Piston 29 articles

At yesterday’s Montreal.rb I presented Nestor, an autotest-like framework. This is it’s official release announcement.

Nestor is different in that it uses an explicit state machine, namely Aaron Pfeifer‘s StateMachine. Nestor also uses Martin Aumont’s Watchr to listen to filesystem events. But the biggest difference is that the default Rails + Test::Unit is a forking test server. Nestor will load the framework classes—ActiveRecord, ActionController, ActionView, plugins and gems—only once. This saves a lot of time on the aggregate versus running rake everytime.

Nestor's state diagram with events denoting success or failure of a run, and states such as green, running_all or run_focused_pending.
Click for larger version

This release of Nestor is 0.2 quality: it’s not ready for large projects. It only supports Rails + Test::Unit, probably doesn’t run on 1.9 or JRuby, but it’s a solid foundation for going further. In the coming days, I will blog on the internals of Nestor and how StateMachine allowed me to add plugins with very little effort.

Installation


1 $ gem install nestor
2 $ cd railsapp
3 $ # edit config/environments/test.rb to set cache_classes to false
4 $ nestor

I already have a plugin that enables Growl notifications. Install and use:


1 $ gem install nestor_growl
2 $ cd railsapp
3 $ nestor start —require nestor/growl

The —require option is where plugins are loaded. This is an Array of files Nestor will require on startup.

Notes

You must set cache_classes to false in test mode for now. This is a limitation of how Rails boots. With cache_classes set to true, Rails will load the controllers and models when it boots. Since this happens before forking, the code under test would never get reloaded. Did I say it was 0.2 quality?

Piston 2.0.8 is here. This is a minor bugfix release:

  • piston status with no path would not check any status. Thanks to Michael Grosser for the heads up;
  • The ActiveSupport gem deprecated require “activesupport” in favor of “active_support”. Thanks for Michael for reporting this as well;
  • Scott Johnson reported and fixed a problem where a git mv would fail because a parent directory was missing.

Thanks to all contributors!

Installing


1 $ gem install piston

See all 29 articles in piston

Tips 22 articles

Quick Puppet Tip

2012-01-24

While testing my Puppet recipes, I usually boot a new instance then apply configuration manually. I have a base configuration, which is extended with other things. I just noticed I could apply a manifest using STDIN:


1 # ( echo "include sysbase" ; echo "include rabbitmq" ) | puppet apply -v

Viva the Unix tradition!

If you do any kind of configuration management, I highly recommend Puppet.

See all 22 articles in tips

Git 16 articles

A while ago, I was wondering how rename tracking in Git worked. I was told that renames didn’t really exist in Git, as Git tracked content, not files themselves. Fair enough.

But, I just stumbled upon something:


1 $ gdc vendor/plugins/acts_as_money/LICENSE
2 diff -git a/vendor/plugins/acts_as_money/LICENSE b/vendor/plugins/acts_as_money/LICENSE
3 index e69de29..a273c73 100644
4 -
- a/vendor/plugins/acts_as_money/LICENSE
5 + b/vendor/plugins/acts_as_money/LICENSE
6 @ -0,0 +1,4 @
7 +one:
8 + user: active
9 + name: name
10 + description: description

Tell me, does that look right? I’ll manually fix that file, but that just decreased my confidence level in Git.

The original LICENSE file was empty, and there apparently was another file (a fixture file) that was empty too, and the latter saw some content added.

Why did this happen? LICENSE was updated as part of a merge—from a branch in which the LICENSE file doesn’t exist.

See all 16 articles in git

Plugins 13 articles

Today, I was building a controller action that was polymorphically finding and associating an object. I could have used a simple dumb solution, such as this:

app/controllers/watches_controller.rb

1 class WatchesController < ApplicationController
2 def create
3 watch</span> = <span class="co">Watch</span>.build(<span class="sy">:watcher</span> =&gt; current_person) <span class="no"> 4</span> <span class="iv">watch.subject = case
5 when params[:person_id]
6 Person.find(params[:person_id])
7 when params[:event_id]
8 Event.find(params[:event_id])
9 else
10 raise ArgumentError, "Don’t know how to handle other keys… #{params.keys.inspect}"
11 end
12 watch</span>.save! <span class="no">13</span> flash[<span class="sy">:notice</span>] = <span class="s"><span class="dl">&quot;</span><span class="k">You're watching </span><span class="il"><span class="idl">#{</span><span class="iv">watch.subject.name}"
14 redirect_to root_url
15 end
16 end

That sucked. Really bad. Why did I have to have a case/switch statement in my controller? Why not use a simpler alternative? I could have gone the full polymorphic route too:

app/controllers/watches_controller.rb

1 class WatchesController < ApplicationController
2 def create
3 watch</span> = <span class="co">Watch</span>.build(<span class="sy">:watcher</span> =&gt; current_person) <span class="no"> 4</span> <span class="no"> <strong>5</strong></span> <span class="c"># For illustration purposes, this is fine, but INSECURE!!!</span> <span class="no"> 6</span> <span class="iv">watch.subject_type = params[:subject_type]
7 watch</span>.subject_id = params[<span class="sy">:subject_id</span>] <span class="no"> 8</span> <span class="iv">watch.save!
9
10 flash[:notice] = "You’re watching #{@watch.subject.name}"
11 redirect_to root_url
12 end
13 end

Then, I remembered that my controller was already a ResourceController implementation. I opened up the code and used this instead:

app/controllers/watches_controller.rb

1 class WatchesController < ResourceController::Base
2 belongs_to :person, :event
3
4 create.before do
5 watch</span>.watcher = current_person <span class="no">6</span> <span class="iv">watch.subject = parent_object
7 end
8 end

I needed to add a bit of infrastructure (some routes and has_many declarations in my models), but those were benefits (I know I’ll need them later anyway). My controller code is simpler, and the intention is clearer, I think.

In the same vein, are you following #standup on Twitter?

See all 13 articles in plugins

Unit-testing 12 articles

When I’m testing admin controllers, I often have tests that follow this form:

test/functional/admin/orders_controller_test.rb

1 class OrdersControllerTest < ActionController::TestCase
2 logged_in_as :active_user do
3 context "on GET to :index" do
4 setup do
5 get :index
6 end
7
8 should_deny_access
9 end
10 end
11
12 not_logged_in do
13 context "on GET to :index" do
14 setup do
15 get :index
16 end
17
18 should_deny_access
19 end
20 end
21 end

Well, this is all Ruby, right? And Ruby has wonderful blocks, and blocks can be passed around…

test/functional/admin/orders_controller_test.rb

1 class OrdersControllerTest < ActionController::TestCase
2 deny_access_tests = lambda do
3 context "on GET to :index" do
4 setup do
5 get :index
6 end
7
8 should_deny_access
9 end
10 end
11
12 logged_in_as :active_user, &deny_access_tests
13 not_logged_in, &deny_access_tests
14 end

This is valid for any block of code that you want to test again and again:

test/functional/admin/orders_controller_test.rb

1 class OrdersControllerTest < ActionController::TestCase
2 successful_index_render = lambda do
3 should_respond_with :success
4 should_render_template "new"
5 should_assign_to :orders
6 end
7
8 logged_in_as :admin do
9 context "", &successful_index_render
10 end
11
12 logged_in_as :sub_admin do
13 context "", &successful_index_render
14 end
15 end

Alternatively, and it might be easier in the end, you could use methods:

test/functional/admin/orders_controller_test.rb

1 class OrdersControllerTest < ActionController::TestCase
2 def self.should_render_successful_index_response
3 should_respond_with :success
4 should_render_template "new"
5 should_assign_to :orders
6 end
7
8 logged_in_as :admin do
9 should_render_successful_index_response
10 end
11
12 logged_in_as :sub_admin do
13 should_render_successful_index_response
14 end
15 end

Note thought that you must define your methods at the top of your test case. Remember that Ruby executes a class definition, so when you suddenly call should_render_successful_index_response, the method definition has to be available, or else Ruby will complain with a NoMethodError.

Ain’t Ruby sweet?

See all 12 articles in unit-testing

Rails 11 articles

Recently, a friend asked me to help him rewrite the Access database they use to track donations into something better. I offered to rewrite as a web app (surprise!), and chose to use Hobo.

I’m not a designer, and it shows. My apps are horrible, and have bad choice of color. Red on blue? Check. Green on orange? Check. Wide margins and misaligned text base? Check. Name any design mistakes, and I can make it happen.

I had seen Hobo used by another developer, and I was impressed with the ActiveRecord extensions. There was also the fact that Hobo has a pleasing design that could serve as an initial draft, until external help can be acquired.

Hobo isn’t really different from Rails: hobo name_of_new_app. Done.

Things I liked immediately: Declarative declaration of fields and validations, automatic generation of migrations, permissions built-in. You know, stuff that covers the 80%

The thing I didn’t like: DRYML. Don’t get me wrong: DRYML is a very nice tool, and I didn’t have any problems understanding the implicit context. All I had was the age-old problem: learning a new API. After 2-3 weeks (very part time), I was more comfortable with the different tags, and how to use them.

I really enjoy writing this:


1 <table-plus fields="this, city.name, toll_free_number, main_number"/>

and receiving this in return:




Click for larger view

Right now, I’ve had a very positive experience with Hobo. Having good documentation helps a lot, and Hobo hasn’t disappointed: http://cookbook.hobocentral.net/

Over the next few days, I’ll discuss Hobo in more details. Hope you follow along!

See all 11 articles in rails

Activerecord 10 articles

The README says it all:


You’re knee deep in a debugger session, and you can’t understand why something’s wrong. You wish you could fire up your application against the test database, but sadly, the process which is running the tests is within a transaction, and thus the actual data is opaque. What can you do?


1 # Somewhere deep in your tests
2 test "the frobble touches the widget" do
3 assert_equal 42, frobble.widget_id
4 end

You’ve been on this assert_equal call for the past hour wondering. Frustration’s been mounting, because you don’t understand why the frobble doesn’t touch the widget. Clearly, there’s something wrong with the fixtures, but you can’t understand what it is. Time to fire up the debugger and dump the data:


1 [814, 823] in test/unit/widget_test.rb
2 814 frobble.save!
3 815 end
4 816
5 817 test "the frobble touches the widget" do
6 818 debugger
7 => 819 assert_equal 42, frobble.widget_id
8 820 end
9 821
10 822 test "the widget touched the frobble in turn" do
11 823 assert widget.touched_by_frobble?
12 test/unit/widget_test.rb:819
13 => 819 assert_equal 42, frobble.widget_id
14 (rdb:112)

Since the data_dumper gem is already declared in your Gemfile (if not, declare it, bundle install, then run your tests again), type:


1 (rdb:112) File.mkdir(Rails.root + "dump")
2 (rdb:113) DataDumper.dump(Rails.root + "dump")

Then, quit your failing tests, and from the trusty command line:


1 $ rails console
2 > DataDumper.load(Rails.root + "dump")
3 > exit
4
5 $ rails server

Any and all data from your test database will be loaded in your development environment. You can now explore your model with your trusty application, to find out what’s really going on.

See all 10 articles in activerecord

Scm 9 articles

Just received an email from Apple today:

From: devbugs@apple.com
Subject: Re: Bug ID 6543251: Git is not available by default on Leopard

Hello François,

This is a follow up to Bug ID# 6543251. After further investigation it has been determined that this is a known issue, which is currently being investigated by engineering. This issue has been filed in our bug database under the original Bug ID# 5404556. The original bug number being used to track this duplicate issue can be found in the State column, in this format: Duplicate/OrigBug#.

Thank you for submitting this bug report. We truly appreciate your assistance in helping us discover and isolate bugs.

Best Regards,

Kit Cheung
Apple Developer Connection
Worldwide Developer Relations

This is a followup to Want Git preinstalled on next Mac OS X?

See all 9 articles in scm

Railsrumble 8 articles

We Are Winners!

2008-11-01

WOW Initially, What Does this Error Mean? had a pretty strong presence in the top leaderboard. As the days went on, we slipped further and further down the scale. We hovered between 10th and 15th on the general leaderboard.

I knew we could sort by different criterias, and I occasionaly looked at the leaderboard by other criterias, but I can’t remember at what ranks we were.

In the end, we won Most Useful. That is a very good thing for us.

Again, congratulations to all teams that participated, and I’m hoping to see another repeat for 2009.

See all 8 articles in railsrumble

Smalltalk 7 articles

I really like Smalltalk’s ifNil: and ifNotNil:, and up to now, I could not use these in Ruby. Fortunately, Bob Hutchison came to the rescue with A Little Unnecessary Smalltalk Envy.

I immediately copied that to XLsuite and wrote a couple of tests. Here is sample of what the code feels like:


1 >> Party.find_by_email_address("sam@gamgee.net").if_not_nil do |party|
2 ?> puts "Found Sam!"
3 >> end
4 => nil

Oh well, no Sam in my database. The code is here:

test/unit/smalltalk_test.rb

1 require File.dirname(FILE) + "/../test_helper"
2
3 class SmalltalkTest < Test::Unit::TestCase
4 setup do
5 block</span> = <span class="co">Proc</span>.new { <span class="pc">true</span> } <span class="no"> 6</span> <span class="r">end</span> <span class="no"> 7</span> <span class="no"> 8</span> context <span class="s"><span class="dl">&quot;</span><span class="k">The nil object</span><span class="dl">&quot;</span></span> <span class="r">do</span> <span class="no"> 9</span> should <span class="s"><span class="dl">&quot;</span><span class="k">yield when calling #if_nil on it</span><span class="dl">&quot;</span></span> <span class="r">do</span> <span class="no"><strong>10</strong></span> assert <span class="pc">nil</span>.if_nil(&amp;<span class="iv">block)
11 end
12
13 should "not yield when calling #if_not_nil on it" do
14 deny nil.if_not_nil(&block</span>) <span class="no"><strong>15</strong></span> <span class="r">end</span> <span class="no">16</span> <span class="r">end</span> <span class="no">17</span> <span class="no">18</span> context <span class="s"><span class="dl">&quot;</span><span class="k">A non nil object</span><span class="dl">&quot;</span></span> <span class="r">do</span> <span class="no">19</span> should <span class="s"><span class="dl">&quot;</span><span class="k">not yield when calling #if_nil on it</span><span class="dl">&quot;</span></span> <span class="r">do</span> <span class="no"><strong>20</strong></span> deny <span class="s"><span class="dl">&quot;</span><span class="dl">&quot;</span></span>.if_nil(&amp;<span class="iv">block)
21 end
22
23 should "yield when calling #if_not_nil on it" do
24 assert "".if_not_nil(&@block)
25 end
26
27 should "pass itself to #if_not_nil" do
28 obj = "abc"
29 assert_same obj, obj.if_not_nil {|o| o}
30 end
31 end
32 end

lib/smalltalk.rb

1 # Copied and adapted from
2 # http://recursive.ca/hutch/2007/11/22/a-little-unnecessary-smalltalk-envy/
3 # Bob Huntchison
4 class Object
5 # yield self when it is non nil.
6 def if_not_nil(&block)
7 yield(self) if block
8 end
9
10 # yield to the block if self is nil
11 def if_nil(&block)
12 end
13 end
14
15 class NilClass
16 # yield self when it is non nil.
17 def if_not_nil(&block)
18 end
19
20 # yield to the block if self is nil
21 def if_nil(&block)
22 yield if block
23 end
24 end

See all 7 articles in smalltalk

Testing 7 articles

I started using Watchr in a couple of projects. An interesting use of Watchr is to check on your tests. I have been using this script with great success so far:

test.watchr

1 def failed
2 system("growlnotify —name adgear-reporting-tests —image /Applications/Mail.app/Contents/Resources/Caution.tiff -m ‘Oops, tests failed’")
3 system("say ‘failed’")
4 end
5
6 def succeeded
7 system("growlnotify —name adgear-reporting-tests -m ‘Green tests’")
8 system("say ‘pass’")
9 end
10
11 watch((lib|test)/.+\.(js|rb)) do |_|
12 system("rake")
13 $?.success? ? succeeded : failed
14 end

Notice I’m using Mac OS X application paths, but it works just fine for me. And the regular “pass” and “failed” messages keep me abreast of what’s going on. But this style of continuous testing only works when your tests take a few seconds at most. When my tests take much longer than that, I start fiddling, opening Google Reader, checking news, whatever.

Your mileage may vary.

See all 7 articles in testing

Scala 6 articles

At Seevibes we use the Twitter Streaming API to harvest what people say about television shows. The Twitter Streaming API docs state that:

Upon a change, reconnect immediately if no changes have occurred for some time. For example, reconnect no more than twice every four minutes, or three times per six minutes, or some similar metric.

From Updating Filter Predicates

The way my system is architected, whenever we change the list of keywords, a ShowDataErased message is sent, followed by a bunch of ShowKeywordsReplaced events, one per show. The system uses CQRS where each change that occurs is reflected in the system through one or more events, hence the two events above.

Here’s my state machine in all it’s gory details:

Seevibes Streaming Harvester State Machine

What the machine does is:

  • When streaming, and time passes, nothing happens;
  • When streaming and we change keywords (either reset or add keyword), we enter a state where we’re pending some changes;
  • When we’re pending, and we change keywords (reset or add again), we stay in the pending state;
  • When we’re pending, and insufficient time has passed, then we must stay in the same state, in case we receive more keyword change events;
  • Finally, if we’re pending and sufficient time has passed, we just go back to the streaming state.

The obvious first test is to start with the fact that when we’re streaming, we must still be streaming. Here’s an implementation of this test.


1 import org.junit.Test
2 import org.scalatest.Assertions
3 import org.joda.time.Instant
4
5 class StreamingStateMachineTest extends Assertions {
6 val KEY2 = Set("b")
7
8 @Test
9 def whenStreaming_andReceiveTick_thenShouldStillBeStreaming() {
10 expect(Streaming(KEY2)) {
11 Streaming(KEY2).tick(new Instant())
12 }
13 }
14 }

For the Rubyists out there, case classes are a kind of class that has a few nice properties:

  • They act like methods, thus we can call the Streaming method (which the compiler turns into a call of Streaming.apply());
  • They have an unapply() method, which enables pattern-matching (which I won’t use in this case);
  • They are serializable, externalizable, clonable, have a free equals() and hashCode() implementation that is correct and consistent with the Java equals() and hashCode() guidelines.

Case classes are an ideal vehicle to transport data with one or more methods. One thing that you must realize when using case classes is that you must use only immutable objects as parameters to your case classes. If you used a mutable object, your case class would suddenly be mutable, and the equals() and hashCode() methods would be useless for you. The Scala compiler has no way to enfore this.

Getting back to our business, the obvious implementation is a no-op:


1 import org.joda.time.ReadableInstant
2 case class Streaming(keywords: Set[String]) {
3 def tick(now: ReadableInstant) = this
4 }

One event down, 6 to go! Let’s change state : go from the streaming state to the pending keywords change state. I’ll start with the simpler case: resetting the keywords list. Here’s an implementation:


1 @Test
2 def whenStreaming_andReceiveReset_thenShouldBePending() {
3 expect(PendingKeywordsChange(Set.empty[String], now)) {
4 Streaming(KEY1).resetKeywords(now)
5 }
6 }

Notice that I’m passing explicit times to my methods. This is to make testing much easier. It doesn’t really bother me to send the current time along. It’s certainly easier than sending a factory which returns the current time, which would add a level of indirection, and be more complicated.

The implementation is also pretty simple:


1 def resetKeywords(now: ReadableInstant) =
2 PendingKeywordsChange(Set.empty[String], now)

And so on for the other implementations of Streaming. The guts of the state machine are really in the PendingKeywordsChange class. First, a test:


1 @Test
2 def whenStreaming_andReceiveAddKeyword_thenShouldBePending() {
3 expect(PendingKeywordsChange(KEY1 + "b", now)) {
4 Streaming(KEY1).addKeyword("b", now)
5 }
6 }
7
8 case class PendingKeywordsChange(keywords: Set[String],
9 lastChangedAt: ReadableInstant) {
10
11 def addKeyword(keyword: String, now: ReadableInstant) =
12 PendingKeywordsChange(newKeywords + keyword, now)
13
14 }

When I wrote the code, I was really excited by how concise and clear my intents shined through the code. The final method I’d like to show is the tick() method on PendingKeywordsChange:


1 @Test
2 def whenPending_andReceiveTick_andInsufficientTimeHasPassed_thenShouldStayPending() {
3 expect(PendingKeywordsChange(KEY1, oneMinuteAgo)) {
4 PendingKeywordsChange(KEY1, oneMinuteAgo).tick(now)
5 }
6
7 expect(PendingKeywordsChange(KEY1, twoMinutesAgo)) {
8 PendingKeywordsChange(KEY1, twoMinutesAgo).tick(now)
9 }
10
11 expect(PendingKeywordsChange(KEY1, twoMinutesAgo)) {
12 PendingKeywordsChange(KEY1, twoMinutesAgo).tick(twoMinutesAgo)
13 }
14 }
15
16 @Test
17 def whenPending_andReceiveTick_andSufficientTimeHasPassed_thenShouldBeStreaming() {
18 expect(Streaming(KEY1, now)) {
19 PendingKeywordsChange(KEY1, twoMinutesAgo.minus(TimeUnit.SECONDS.toMillis(1))).tick(now)
20 }
21 }
22
23 // in class PendingKeywordsChange
24 def nextChangeAt = lastChangedAt.toInstant.plus(TimeUnit.MINUTES.toMillis(2))
25
26 def tick(now: ReadableInstant = new Instant()) =
27 if (now.isAfter(nextChangeAt))
28 Streaming(newKeywords)
29 else
30 this

The final part is where and how I regularly call the tick event. Because the rest of my infrastructure is tied to Akka actors, I used the Akka scheduler API to send a message to an actor, which hid my state machine behind a nice and consistent facade:


1 import akka.actor.Actor
2
3 case object Tick
4 case object ResetKeywords
5 case class AddKeyword(keyword: String)
6
7 trait State {
8 def resetKeywords(now: ReadableInstant): State
9 def addKeyword(keyword: String, now: ReadableInstant): State
10 def tick(now: ReadableInstant): State
11 def keywords: Set[String]
12 }
13
14 case class Streaming(keywords: Set[String],
15 lastChangedAt: ReadableInstant) extends State {
16 def resetKeywords(now: ReadableInstant) =
17 PendingKeywordsChange(Set.empty[String], now)
18
19 def addKeyword(keyword: String, now: ReadableInstant) =
20 PendingKeywordsChange(Set(keyword), now)
21
22 def tick(now: ReadableInstant) = this
23 }
24
25 case class PendingKeywordsChange(keywords: Set[String],
26 lastChangedAt: ReadableInstant) extends State {
27 def resetKeywords(now: ReadableInstant) =
28 PendingKeywordsChange(Set.empty[String], now)
29
30 def addKeyword(keyword: String, now: ReadableInstant) =
31 PendingKeywordsChange(keywords + keyword, now)
32
33 def nextChangeAt = lastChangedAt.toInstant.plus(TimeUnit.MINUTES.toMillis(2))
34
35 def tick(now: ReadableInstant = new Instant()) =
36 if (now.isAfter(nextChangeAt))
37 Streaming(keywords)
38 else
39 this
40 }
41
42 class KeywordsStateMachine extends Actor {
43 private var state: State = Streaming(Set.empty[String], new Instant())
44 private var currentKeywords = Set.empty[String]
45 private val stream = … // Twitter4J Streaming API
46
47 def receive = {
48 case Tick =>
49 state = state.tick(new Instant)
50 if (state.keywords != currentKeywords)
51 // reset stream with new keywords
52
53 case AddKeyword(keyword) =>
54 state = state.addKeyword(keyword, new Instant())
55
56 case ResetKeywords =>
57 state = state.resetKeywords(new Instant())
58 }
59 }
60
61 object Main {
62 def main(args : Array[String]) {
63 val keywordsStateMachine =
64 Actor.actorOf[KeywordsStateMachine].start()
65
66 // Send the Tick message to the keywordsStateMachine now,
67 // and every minute thereafter
68 Scheduler.schedule(keywordsStateMachine, Tick, 0, 1, TimeUnit.MINUTES)
69 }
70 }

A few things contributed to the clarity of the implementation:

  • The Joda Time library is really easy to use and understand. You don’t need to use java.util.Date or java.util.Calendar : ditch them as soon as you can;
  • Scala’s case classes reduced boilerplate: no new operator in sight;
  • Public by default enhances clarity by removing keywords where they don’t matter;
  • Libraries, yes, but not when something simple is required. Clarity of implementation before reuse.

Note that this state machine does not have to deal with dropped connections, or rate limiting or anything of the sort, because it’s all hidden behind Twitter4J’s interface. The resulting state machine is much easier to understand and reason about.

In the interest of fun, profit and learning, I’ve made available a GitHub repository with similar code to what’s here at streaming-state-machine. The code is different because it doesn’t actually connect to Twitter, and thus only needs to demonstrate how changing the states works. The time scale was changed from minutes to seconds.

See all 6 articles in scala

Seaside 6 articles

I have worked some more on my home budget planner application. The current version is implemented on Seaside, and it’s doing great. I haven’t had as much time as I wanted, but it’s coming along nicely. I have some formal training in accounting practices, and I treat my house as being a business: we have income and expense accounts, equity for ourselves, and assets (our bank account) as well as liability accounts (credit cards and bank loans).



BudgetApp’s administration tab. Click for larger view.

This is the administration tab. This is where you add and change accounts. Not much more to show here.



BudgetApp’s budget tab. Click for larger view.

The budget tab. This is where you actually set your budget targets for the month. Historical data is kept around, and you can immediately see when your budget is under the actual value.



BudgetApp’s real tab. Click for larger view.

This is the least polished of the tabs yet, and ironically, this is where most of the work is going to be done. I’ll need to use the application for a bit before I can determine the exact interface I want. I’m thinking of having a couple of panels that will allow the user to say what kind of transaction occurred: paid, bought, reimbursed, transferred, etc.

That’s the state of affairs at revision 9 on the Monticello repository.

Differences with Rails

I haven’t actually started doing any work on the Rails side of things, but there is one thing I did notice: I find it easier to segregate my work in change sets in the file world versus when working in the image-world. For example, revision 8 includes changes to a couple of classes, and none are related: stylistic changes in the budget and admin tabs, plus my initial stab at the real tab. Had I been using Rails, I would have committed a couple of files here and there multiple times, and that would be it.

I am aware of Monticello’s “add to current change set” and “remove from current change set”, but have not dared using them yet. I’m not exactly sure what these options will do, and most importantly, I am afraid of losing work. That probably won’t happen, but there’s this nagging feeling deep down…

Anyway, next step is to generate real transactions from the real tab. More on this next week !

See all 6 articles in seaside

Ruby 6 articles

I was meta-meta-programming, and had an issue when an #included method was called:


1 anon = Module.new
2 anon.class_eval do
3 def self.included(base)
4 debugger
5 if some_method then
6 # code
7 else
8 # more code
9 end
10 end
11 end

The code above resulted in:


1 /Users/francois/Projects/project/lib/extensions.rb:214:in `included’: undefined local variable or method `some_method for #<Module:0×103305b20> (NameError)
2 from /Users/francois/Projects/project/lib/extensions.rb:245:in `include

3 from /Users/francois/Projects/project/lib/extensions.rb:245:in `send’
4
5

The debugger statement above just wouldn’t take: Ruby ran right over it. I happened to look at ruby-debug’s Rubygem spec file:


1 - !ruby/object:Gem::Specification
2 name: ruby-debug
3
4 executables:
5 – rdebug
6

Oh, had never noticed the executables before… Sure enough, I managed to run under debugger control immediately:


1 $ rdebug -I test test/functional/ad_spots_controller_test.rb [-4, 5] in /Users/francois/Projects/bloom/adgear-admin/test/functional/ad_spots_controller_test.rb
2 => 1 require "test_helper"
3 2
4 3 class AdSpotsControllerTest < ActionController::TestCase
5 4
6 5 def setup
7 /Users/francois/Projects/bloom/adgear-admin/test/functional/ad_spots_controller_test.rb:1
8 require "test_helper"
9 (rdb:1)

From there, I was able to use (C)ontinue to end up on my debugger statement. Happy times ensued!

For the curious, the NoMethodError is because #some_method is defined on base, not on self. self in this context is the included module, while base is the place where we’re including it into.

See all 6 articles in ruby

Rubyfringe 5 articles

Tearful Goodbye, by chaim zvi.

I had so much fun at RubyFringe that I don’t want to go! I’m in the Metropolitan lobby, and writing this. I almost have a tear in my eyes. What fond recollections I will cherish for a long time to come.

Thanks to the Unspace crew (Pete, Megan, Hampton and the others I haven’t thanked personally but who did an amazing job) for making this event.

This was my first conference ever, and it was great. As Pete said, I’m happy we took your conference virginity.

Photo by chaim zvi, Tearful Goodbye

See all 5 articles in rubyfringe

Svn 5 articles

They just work. Ain’t that cool ?


1 $ ln -s a b
2 $ git add b
3 $ git commit
4 $ git svn dcommit
5 $ cd ../svn-wc
6 $ svn up
7 $ svn pl -v b
8 Properties on ‘b’:
9 svn:special : *
10 $ ls -l
11 lrwxr-xr-x 1 francois staff 14 29 mai 15:10 a
12 lrwxr-xr-x 1 francois staff 14 29 mai 15:10 b -> a

Did you know symlinks were supported in Subversion since 1.1.0 ? I certainly didn’t remember.

See all 5 articles in svn

Aws 5 articles

Please grab cliaws from Rubyforge:


1 $ gem install cliaws

What is Cliaws?

Cliaws is a replacement for the Amazon EC2 API tools, but uses Ruby, and thus does not suffer from a long boot time. Cliaws is also easier to setup:


1 $ AWS_ACCESS_KEY_ID=# Your Access ID
2 $ AWS_SECRET_ACCESS_KEY=# Your secret ID
3 $ # Setup is done, enjoy!
4 $ clis3 list YOUR_BUCKET
5 $ cliec2 launch AMI —keypair KEYNAME

Please view the README for some details. The implementation is still the best place to get information about the options you can pass.

See all 5 articles in aws

Release 5 articles

Have you ever wished you could access your Amazon Simple Queue Service from the command line ? Now you can:


1 $ AWS_ACCESS_KEY_ID=<your access key>
2 $ AWS_SECRET_ACCESS_KEY="<the secret access key>"
3
4 $ clisqs create my-queue
5 Queue my-queue was created.
6
7 $ clisqs list
8 my-queue
9
10 $ clisqs push —data "this is the message" my-queue
11 Pushed 19 bytes to queue my-queue
12
13 $ cat README.txt | clisqs push my-queue
14 Pushed 2687 bytes to queue my-queue
15
16 $ clisqs push my-queue README.txt
17 Pushed 2687 bytes to queue my-queue
18
19 $ clisqs size my-queue
20 3
21
22 $ clisqs pop my-queue
23 this is the message
24
25 $ clisqs delete —force my-queue
26 Queue my-queue was deleted.

Installation


1 $ sudo gem install cliaws

Direct-code access


1 require "rubygems"
2 require "cliaws"
3
4 Cliaws.sqs.push("my-queue", "the data")
5 the_size = Cliaws.sqs.size("my-queue")
6 the_message = Cliaws.sqs.pop("my-queue")

S3 ?

This gem also works with S3. See my prior release announcement: Cliaws: command-line access to S3

See all 5 articles in release

Mephisto 4 articles

Yesterday (Tuesday May 20th, 2008), I presented at Montreal on Rails. I made a short and sweet presentation on Mephisto, and how I refactored it to support both Akismet and Defensio.

You can grab the slides for “Refactoring to Patterns: How Mephisto went from a single engine Lada to a multi-engine jet fighter”/2008/05/21/refactoring-to-patterns.pdf (PDF).

References

Design Patterns

Other interesting patterns that I used in Mephisto, which I briefly talked about, but haven’t mentioned in the slides at all:

Refactoring

  • Refactoring to Patterns

Other things I talked about

See all 4 articles in mephisto

Mor 4 articles

Yesterday (Tuesday May 20th, 2008), I presented at Montreal on Rails. I made a short and sweet presentation on Mephisto, and how I refactored it to support both Akismet and Defensio.

You can grab the slides for “Refactoring to Patterns: How Mephisto went from a single engine Lada to a multi-engine jet fighter”/2008/05/21/refactoring-to-patterns.pdf (PDF).

References

Design Patterns

Other interesting patterns that I used in Mephisto, which I briefly talked about, but haven’t mentioned in the slides at all:

Refactoring

  • Refactoring to Patterns

Other things I talked about

See all 4 articles in mor

Tools 4 articles

The latest release of Piston provides you with the ability to switch the upstream repository locations without losing any history.

For example:


1 $ piston switch http://dev.rubyonrails.org/svn/rails/tags/rel_1-2-1 vendor\rails
2 Processing ‘vendor\rails’…
3 Fetching remote repository’s latest revision and UUID
4 Restoring remote repository to known state at r6010
5 Updating remote repository to http://dev.rubyonrails.org/svn/rails/tags/rel_1-2-1@5990
6 Processing adds/deletes
7 Removing temporary files / folders
8 Updating Piston properties
9 Updated to r5990 (663 changes)

Piston 1.3.0 also shows the revision number of locked directories.

Installation

As usual, nothing could be simpler:


1 $ sudo gem install —include-dependencies piston

See all 4 articles in tools

Cliaws 4 articles

Today sees a new release of Cliaws. This is a simple library to interact with S3 (and other services) through the command-line. This library is very similar to Amazon’s own ec2-* scripts, except this library is written in Ruby.

Interacting with S3 should be no harder than:


1 $ clis3 put my-local-file my-bucket/my-s3-file

Put many files at once:


1 $ clis3 put my-local-file0 my-local-file1 my-bucket/my-s3-directory/

Put an environment variable:


1 $ clis3 put —data $MY_DATA my-bucket/my-env-value

Put stdin too!


1 $ tar czfv /var/cache/mylvmbackup/backup | clis3 put – my-bucket/my-backup/backup-20080825-000000.tar.gz

All of this functionnality is available from with Ruby too:


1 require "rubygems"
2 require "cliaws"
3
4 Cliaws.s3.put("this is the content", "my-bucket/my-s3-file")
5 File.open("my-local-file", "rb") do |io|
6 Cliaws.s3.put(io, "my-bucket/my-s3-file")
7 end

Give the RubyForge gem servers a couple of hours to refresh themselves, and then enjoy!

See all 4 articles in cliaws

Version-control 4 articles

Okay, I have a use case which I think shouldn’t be too hard. Jean-François, my Montreal on Rails buddy, gave me a good hand, but I think there’s some interaction going on between Github and git-svn that causes friction…

I have a new project which I want to track using Git, but I also want to keep a push-only Subversion repository on Rubyforge.

So, I started like any old project:


1 $ svn checkout svn+ssh://fbos@rubyforge.org/var/svn/theproject
2 $ svn mkdir tags branches
3 $ newgem theproject
4 $ mv theproject trunk
5 $ svn add trunk
6 $ svn commit -m "New project layout"

Then, I imported that into a fresh Git repository:


1 $ git svn clone svn+ssh://fbos@rubyforge.org/var/svn/theproject -T trunk -t tags -b branches theproject

I then started coding and commiting using Git. All was well, and I was happy. Then, I pushed to Subversion, like any good developer should:


1 $ git svn dcommit

That worked well enough. Then, I was ready to have a public Git repository. I went to Github, where I have an account, and created a new repository. I then did:


1 $ git remote add github git@github.com:francois/theproject.git
2 $ git push github master

That gave me a couple of errors:


1 error: remote ‘refs/heads/master’ is not a strict subset of local ref
2 ‘refs/heads/master’. maybe you are not up-to-date and need to pull first?
3 error: failed to push to ‘git@github.com:francois/theproject.git’

After some discussion with Jean-François, he told me to have a separate branch on which to do development, and merge to master when I am ready to dcommit. So, I did:


1 $ git checkout -b mainline
2 # code again
3 $ git checkout master
4 $ git merge mainline
5 $ git svn dcommit

That works well enough. Now I want to push my mainline to Github. So I do:


1 $ git remote add github git@github.com:francois/theproject.git
2 $ git push github mainline
3 $ git push github mainline
4 updating ‘refs/heads/mainline’
5 from 0000000000000000000000000000000000000000
6 to 31d6eee71b00f829b8568937ab3adaaa8831205c
7 Generating pack…
8 Done counting 201 objects.
9 Deltifying 201 objects…
10 100% (201/201) done
11 Writing 201 objects…
12 100% (201/201) done
13 Total 201 (delta 100), reused 0 (delta 0)
14 refs/heads/mainline: 0000000000000000000000000000000000000000 -> 31d6eee71b00f829b8568937ab3adaaa8831205c

Great! It works. Except… The repository page on Github still shows I need to create the repository. Just for fun, without doing anything else I created a new empty Git repository, touched README, added that, remoted Github and pushed. Voilà, Github showed me the repository, with README being added.

Can anyone shed some light on this ? Here’s a reproduction recipe:


1 $ cat repro.sh
2 #!/bin/sh
3
4 rm -rf repos wc theproject
5 svnadmin create repos
6 svn checkout file:///`pwd`/repos wc
7 cd wc
8 svn mkdir tags branches
9 newgem theproject
10 mv theproject trunk
11 svn add trunk
12 svn commit -m "Initial revision"
13 cd ..
14 git svn clone file:///`pwd`/repos theproject
15 cd theproject
16 echo "The new revision" > README
17 git add README
18 git commit -a -m "New README content"
19 git checkout -b mainline
20 git checkout master
21 git svn dcommit
22 git checkout mainline
23 echo "More content" >> README
24 git commit -a -m "More README goodness"
25 git remote add github git@github.com:francois/theproject.git
26 git push github mainline
27 git checkout master
28 git merge mainline
29 git svn dcommit

And here’s a sample bash +x run:


1 + rm rf repos wc theproject
2 + svnadmin create repos
3 + pwd
4 + svn checkout file:////home/francois/src/repro/repos wc
5 Checked out revision 0.
6 + cd wc
7 + svn mkdir tags branches
8 A tags
9 A branches
10 + newgem theproject
11 create
12 create config
13 create doc
14 create lib
15 create log
16 create script
17 create tasks
18 create test
19 create tmp
20 create lib/theproject
21 create History.txt
22 create License.txt
23 create Rakefile
24 create README.txt
25 create setup.rb
26 create lib/theproject.rb
27 create lib/theproject/version.rb
28 create config/hoe.rb
29 create config/requirements.rb
30 create log/debug.log
31 create tasks/deployment.rake
32 create tasks/environment.rake
33 create tasks/website.rake
34 create test/test_helper.rb
35 create test/test_theproject.rb
36 dependency install_website
37 create website/javascripts
38 create website/stylesheets
39 exists script
40 exists tasks
41 create website/index.txt
42 create website/index.html
43 create script/txt2html
44 force tasks/website.rake
45 dependency plain_theme
46 exists website/javascripts
47 exists website/stylesheets
48 create website/template.rhtml
49 create website/stylesheets/screen.css
50 create website/javascripts/rounded_corners_lite.inc.js
51 dependency install_rubigen_scripts
52 exists script
53 create script/generate
54 create script/destroy
55 create Manifest.txt
56 readme readme
57 Important
58 =====
59
60 * Open config/hoe.rb
61 * Update missing details (gem description, dependent gems, etc.)
62 + mv theproject trunk
63 + svn add trunk
64 A trunk
65 A trunk/Manifest.txt
66 A trunk/History.txt
67 A trunk/doc
68 A trunk/setup.rb
69 A trunk/tmp
70 A trunk/test
71 A trunk/test/test_theproject.rb
72 A trunk/test/test_helper.rb
73 A trunk/tasks
74 A trunk/tasks/deployment.rake
75 A trunk/tasks/website.rake
76 A trunk/tasks/environment.rake
77 A trunk/lib
78 A trunk/lib/theproject.rb
79 A trunk/lib/theproject
80 A trunk/lib/theproject/version.rb
81 A trunk/script
82 A trunk/script/txt2html
83 A trunk/script/destroy
84 A trunk/script/generate
85 A trunk/Rakefile
86 A trunk/website
87 A trunk/website/stylesheets
88 A trunk/website/stylesheets/screen.css
89 A trunk/website/javascripts
90 A trunk/website/javascripts/rounded_corners_lite.inc.js
91 A trunk/website/index.txt
92 A trunk/website/template.rhtml
93 A trunk/website/index.html
94 A trunk/log
95 A trunk/log/debug.log
96 A trunk/config
97 A trunk/config/requirements.rb
98 A trunk/config/hoe.rb
99 A trunk/License.txt
100 A trunk/README.txt
101 + svn commit -m ‘Initial revision’
102 Adding branches
103 Adding tags
104 Adding trunk
105 Adding trunk/History.txt
106 Adding trunk/License.txt
107 Adding trunk/Manifest.txt
108 Adding trunk/README.txt
109 Adding trunk/Rakefile
110 Adding trunk/config
111 Adding trunk/config/hoe.rb
112 Adding trunk/config/requirements.rb
113 Adding trunk/doc
114 Adding trunk/lib
115 Adding trunk/lib/theproject
116 Adding trunk/lib/theproject/version.rb
117 Adding trunk/lib/theproject.rb
118 Adding trunk/log
119 Adding trunk/log/debug.log
120 Adding trunk/script
121 Adding trunk/script/destroy
122 Adding trunk/script/generate
123 Adding trunk/script/txt2html
124 Adding trunk/setup.rb
125 Adding trunk/tasks
126 Adding trunk/tasks/deployment.rake
127 Adding trunk/tasks/environment.rake
128 Adding trunk/tasks/website.rake
129 Adding trunk/test
130 Adding trunk/test/test_helper.rb
131 Adding trunk/test/test_theproject.rb
132 Adding trunk/tmp
133 Adding trunk/website
134 Adding trunk/website/index.html
135 Adding trunk/website/index.txt
136 Adding trunk/website/javascripts
137 Adding trunk/website/javascripts/rounded_corners_lite.inc.js
138 Adding trunk/website/stylesheets
139 Adding trunk/website/stylesheets/screen.css
140 Adding trunk/website/template.rhtml
141 Transmitting file data ……………………
142 Committed revision 1.
143 + cd ..
144 +
pwd
145 + git svn clone file:////home/francois/src/repro/repos theproject
146 Initialized empty Git repository in .git/
147 A trunk/History.txt
148 A trunk/test/test_helper.rb
149 A trunk/test/test_theproject.rb
150 A trunk/License.txt
151 A trunk/log/debug.log
152 A trunk/Rakefile
153 A trunk/setup.rb
154 A trunk/website/template.rhtml
155 A trunk/website/index.txt
156 A trunk/website/javascripts/rounded_corners_lite.inc.js
157 A trunk/website/index.html
158 A trunk/website/stylesheets/screen.css
159 A trunk/Manifest.txt
160 A trunk/script/txt2html
161 A trunk/script/destroy
162 A trunk/script/generate
163 A trunk/config/requirements.rb
164 A trunk/config/hoe.rb
165 A trunk/tasks/deployment.rake
166 A trunk/tasks/website.rake
167 A trunk/tasks/environment.rake
168 A trunk/lib/theproject/version.rb
169 A trunk/lib/theproject.rb
170 A trunk/README.txt
171 W: empty_dir: branches
172 W: +empty_dir: tags
173 W: +empty_dir: trunk/doc
174 W: +empty_dir: trunk/tmp
175 r1 = 01d09dc543711efb5bbd39e71ea2d1fe12516926 (git-svn)
176
177 Checked out HEAD:
178 file:////home/francois/src/repro/repos r1
179 + cd theproject
180 + echo ‘The new revision’
181 + git add README
182 + git commit -a -m ‘New README content’
183 Created commit 16eaad5: New README content
184 1 files changed, 1 insertions(
), 0 deletions(
)
185 create mode 100644 README
186 + git checkout b mainline
187 Branch mainline set up to track local branch refs/heads/master.
188 Switched to a new branch "mainline"
189 + git checkout master
190 Switched to branch "master"
191 + git svn dcommit
192 A README
193 Committed r2
194 A README
195 r2 = 5b2c01dde36111a37f942d9fb6670689089ad9ed (git-svn)
196 No changes between current HEAD and refs/remotes/git-svn
197 Resetting to the latest refs/remotes/git-svn
198 + git checkout mainline
199 Switched to branch "mainline"
200 + echo ‘More content’
201 + git commit -a -m ‘More README goodness’
202 Created commit 49df2b5: More README goodness
203 1 files changed, 1 insertions(+), 0 deletions(
)
204 + git remote add github git@github.com:francois/theproject.git
205 + git push github mainline
206 error: remote ‘refs/heads/mainline’ is not a strict subset of local ref ‘refs/heads/mainline’. maybe you are not up-to-date and need to pull first?
207 error: failed to push to ‘git@github.com:francois/theproject.git’
208 + git checkout master
209 Switched to branch "master"
210 + git merge mainline
211 Auto-merged README
212 CONFLICT (add/add): Merge conflict in README
213 Automatic merge failed; fix conflicts and then commit the result.
214 + git svn dcommit
215 No changes between current HEAD and refs/remotes/git-svn
216 Resetting to the latest refs/remotes/git-svn
217 README: needs update

See all 4 articles in version-control

Engines 4 articles

I just got hit by this famous error again. And I am not the only one:

Now, I know I should put my patches where my mouth is. And that’s exactly what I’m trying to do here. My application is on the 1.2 branch, and I get this error if I have the Response Logger plugin loaded. Trying again with Edge Rails, I again get the error. If I try with WEBrick or Mongrel, same error. Loading the console produces the same error. Using Windows or Linux changes nothing: same error.

The exact backtrace is:


1 $ ruby script\console
2 Loading development environment.
3 ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:249:in `load_missing_constant’: Expected ./script/../config/../config/../vendor/plugins/response_logger/lib/response_logger.rb to define ResponseLogger (LoadError)
4 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:452:in `const_missing’
5 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:464:in `const_missing’
6 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:260:in `load_missing_constant’
7 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:452:in `const_missing’
8 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:260:in `load_missing_constant’
9 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:452:in `const_missing’
10 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:260:in `load_missing_constant’
11 from ./script/../config/../config/../vendor/rails/activerecord/lib/../../activesupport/lib/active_support/dependencies.rb:468:in `const_missing’
12 … 14 levels…
13 from C:/ruby/lib/ruby/1.8/irb/init.rb:250:in `load_modules’
14 from C:/ruby/lib/ruby/1.8/irb/init.rb:21:in `setup’
15 from C:/ruby/lib/ruby/1.8/irb.rb:54:in `start’
16 from C:/ruby/bin/irb.bat:20

In this application, I am using Engines, along with LoginEngine. You really need the right version of Engines with the right version of Rails for things to work out well: Rails 1.2 requires Engines 1.2.

See all 4 articles in engines

Windows 4 articles

If you need to run Piston on Windows, BoxCycle wrote some information about it at: Rails Plugin Updates, SVN, and Piston 2.0.2 on Windows

See all 4 articles in windows

Security 4 articles

Well, it seems I jumped the gun. In Security issue in Liquid::Template, I thought I had a found a problem with the Liquid template engine. Instead, I should have looked more closely at what I do:

app/controllers/pages_controller.rb

1 class PagesController < ApplicationController
2 def show
3 # …
4 render(:inline => @page.render, :layout => false)
5 end
6 end

The details can be found at #render on the Ruby on Rails API. Seems like it’s time for us to switch to using render :text.

I am sorry for any scare I caused. If I had run a separate test case, I’d have immediately seen I was in error, and not Liquid.

See all 4 articles in security

Git-svn 3 articles

They just work. Ain’t that cool ?


1 $ ln -s a b
2 $ git add b
3 $ git commit
4 $ git svn dcommit
5 $ cd ../svn-wc
6 $ svn up
7 $ svn pl -v b
8 Properties on ‘b’:
9 svn:special : *
10 $ ls -l
11 lrwxr-xr-x 1 francois staff 14 29 mai 15:10 a
12 lrwxr-xr-x 1 francois staff 14 29 mai 15:10 b -> a

Did you know symlinks were supported in Subversion since 1.1.0 ? I certainly didn’t remember.

See all 3 articles in git-svn

Sql 3 articles

A great time was had by all!

Seriously, the Scala workshop was very cool. The actor model was new to almost all attendants and needed some explaining. Looking back, I should have explained a bit more about how the this works. From what I saw, nobody stumbled upon the Scala syntax that much. The people that were there were all experienced programmers, thus may not have needed much hand-holding regarding syntax.

Martin Provencher, organizer of Montreal.rb, and his teammate, Olivier Melcher, wrote an actor-based histogram producing word counter that counted which programming languages were most spoken of. In their tests, Ruby and PHP were the most frequent ones.

The SQL Workshop was more difficult for all. The first exercise was much harder than I anticipated. The first question was:

Calculate the top 5 shows, ordered by social impressions, for the period between 2011-10-17 and 2011-10-23. Return an answer under 3 seconds.

First, people had to get familiar with the data model. I asked people to get the data in under 3 seconds, thus was more of an optimization problem, rather than just getting the correct query. I allowed about 40 minutes for this exercise, and most of the time was used in getting a correct query. Nobody had time to get to the optimization part. Looking back, I’d make this question be two parts: first write the query, then optimize it, or provide the correct query and let people optimize it.

I hope everybody learned a lot by coming to the hackaton.

A big “Thank You” to the event’s sponsors for the beer and pizza:

Notes for Hackaton organizers

If you can, provide a starter application or package. All attendees were happy they could get up and running with little to no fuss. It saved a lot of time because they didn’t have to resolve dependency nightmares.

Move around the room, look over everybody’s shoulders. Without fail, when I walked up to a team, they’d have a question for me.

Speaking of teams, it’s useful for people to work in teams. I’d say this harkens back to Pair Programming with a Novice / Novice pair. Two novices working together can get further along than a single novice. One of the members will have an insight that the other member didn’t have, and both can move forward from there.

See all 3 articles in sql

Migrations 3 articles

I recently hit upon the Enhanced Migrations plugin by Revolution on Rails. Works great when you develop on branches. The #dump_schema_information method of ActiveRecord::ConnectionAdapters::SchemaStatements only dumps the most recently migration file. Since each migration is now a separate entry in the migrations_info table, we can’t report only the latest one.

To this end, I generated the following diff:


1 $ svn diff vendor
2 Index: vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb
3 ===============
4 - vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb (revision 7767)
5 + vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb (working copy)
6 </span><span class="er"> -58,8 +58,8 </span><span class="er">
7
8 ActiveRecord::ConnectionAdapters::SchemaStatements.send(:define_method, :dump_schema_information) do
9 begin
10if (current_schema = ActiveRecord::Migrator.current_version) > 0
11return "INSERT INTO #{ActiveRecord::Migrator.schema_info_table_name} VALUES (#{current_schema}, NOW"
12 + select_all("SELECT * FROM #{ActiveRecord::Migrator.schema_info_table_name} ORDER BY created_at, id").map do |migration|
13 + "INSERT INTO #{ActiveRecord::Migrator.schema_info_table_name} VALUES;\n"
14 end
15 rescue ActiveRecord::StatementInvalid
16 # No Schema Info

Hope this is useful for other people.

See all 3 articles in migrations

Scope 3 articles

Ain’t Release Early, Release Often wonderful ? I just received a patch from Evgeniy Pirogov that makes the failing test from the AutoScope plugin work.

Evgeniy’s patch is available here, but you should simply svn or piston update to get the latest version, since I eagerly accepted Evgeniy’s patch.

Thank you, Evgeniy for your work.

See all 3 articles in scope

Ubuntu 3 articles

I just posted about my problem with the English US keyboard not having a pipe character ? Well, I removed the French Canadian keyboard layout from my configuration, and now I have my pipe character back. That’s not very nice, since I won’t be able to type in French anymore, but at least, I can program again.

Thanks to dvorak keyboard layout but keyboard shortcuts are in qwrty for the idea.

See all 3 articles in ubuntu

Autotest 3 articles

At yesterday’s Montreal.rb I presented Nestor, an autotest-like framework. This is it’s official release announcement.

Nestor is different in that it uses an explicit state machine, namely Aaron Pfeifer‘s StateMachine. Nestor also uses Martin Aumont’s Watchr to listen to filesystem events. But the biggest difference is that the default Rails + Test::Unit is a forking test server. Nestor will load the framework classes—ActiveRecord, ActionController, ActionView, plugins and gems—only once. This saves a lot of time on the aggregate versus running rake everytime.

Nestor's state diagram with events denoting success or failure of a run, and states such as green, running_all or run_focused_pending.
Click for larger version

This release of Nestor is 0.2 quality: it’s not ready for large projects. It only supports Rails + Test::Unit, probably doesn’t run on 1.9 or JRuby, but it’s a solid foundation for going further. In the coming days, I will blog on the internals of Nestor and how StateMachine allowed me to add plugins with very little effort.

Installation


1 $ gem install nestor
2 $ cd railsapp
3 $ # edit config/environments/test.rb to set cache_classes to false
4 $ nestor

I already have a plugin that enables Growl notifications. Install and use:


1 $ gem install nestor_growl
2 $ cd railsapp
3 $ nestor start —require nestor/growl

The —require option is where plugins are loaded. This is an Array of files Nestor will require on startup.

Notes

You must set cache_classes to false in test mode for now. This is a limitation of how Rails boots. With cache_classes set to true, Rails will load the controllers and models when it boots. Since this happens before forking, the code under test would never get reloaded. Did I say it was 0.2 quality?

Piston 2.0.8 is here. This is a minor bugfix release:

  • piston status with no path would not check any status. Thanks to Michael Grosser for the heads up;
  • The ActiveSupport gem deprecated require “activesupport” in favor of “active_support”. Thanks for Michael for reporting this as well;
  • Scott Johnson reported and fixed a problem where a git mv would fail because a parent directory was missing.

Thanks to all contributors!

Installing


1 $ gem install piston

See all 3 articles in autotest

Postgresql 3 articles


1 # JRuby 1.6.0 required
2 require "sequel"
3 require "jdbc/postgresql"
4
5 DB = Sequel.connect "jdbc:postgresql://127.0.0.1:5432/mydb"
6
7 table = DB[:mytable]
8 time = Benchmark.measure do
9 (1..10000).each do |n|
10 table.insert(:a => n, :b => 2*n)
11 end
12 end
13
14 puts time

How much time do you think this is going to take? This is the most inefficient way to insert many rows to a database. Remember each call to #insert will do a round trip to the database. Even if your round trip time is 1ms, you’ll still pay for 10 seconds of round trip time, time which your program could be doing something much more useful, such as generating revenue (somehow).

Instead, you should use the bulk copy feature of your database engine. In my case, that’s PostgreSQL. Since I’m using JRuby, I have to turn to the JDBC world, but that’s all right: everything has been implemented already, by someone, somewhere. I’ll refer you to the relevant pages:

And the relevant code would be:


1 # JRuby 1.6.0 required
2 require "sequel"
3 require "jdbc/postgresql"
4 require "java"
5
6 DB = Sequel.connect "jdbc:postgresql://127.0.0.1:5432/mydb"
7
8 time = Benchmark.measure do
9 DB.synchronize do |connection|
10 copy_manager = org.postgresql.copy.CopyManager.new(connection)
11 stream = copy_manager.copy_in("COPY mytable(a, b) FROM STDIN WITH CSV")
12
13 begin
14 (1..10000).each do |n|
15 # Don’t forget we’re streaming CSV data, thus each row/line MUST be terminated with a newline
16 row = "#{n},#{2*n}\n".to_java_bytes
17 stream.write_to_copy(row, 0, row.length)
18 end
19 rescue
20 stream.cancel_copy
21 raise
22 else
23 stream.end_copy
24 end
25 end
26 end
27
28 puts time

This will execute a single round trip to the database server: you’ll pay the latence cost only once.

On an unrelated note, this is the first time ever I use an else clause on a begin/rescue. If an exception is raised, we want to cancel the copy (the rescue clause), but on the other hand, if nothing is raised, we want to end the copy (the else clause). One or the other must happen, but not both.

If you’re curious what difference bulk copying makes, here are the benchmark results:

10000 INSERT statements
  7.012000   0.000000   7.012000 (  7.012000)

1 COPY FROM STDIN statement
  0.848000   0.000000   0.848000 (  0.848000)

The numbers speak for themselves: 8× faster. Not too shabby, and remember this ratio will simply increase as the number of rows increases.

See all 3 articles in postgresql

Fixtures 3 articles

Bob Silva posted Testing Gotchas in Rails in mid-October. I had fallen across this error myself a few times.

Well, today I had a huge code base I needed to check. I wrote the following Rake task:

lib/tasks/find_missing_fixtures.rake

1 # Find missing fixtures declarations from Rails tests
2 # Code by Francois Beausoleil (francois@teksol.info)
3 # Released in the public domain. Do as you wish.
4 desc "Finds missing fixtures declarations from your tests"
5 task :find_missing_fixtures do
6 state = :find_test_case
7 test_case = nil
8
9 Dir[test//_test.rb].each do |file|
10 File.open(file, r) do |f|
11 f.each do |line|
12 case state
13 when :find_fixture
14 case line
15 when /def test_/
16 printf "%s: %s\n", file, test_case
17 state = :find_test_case
18 when /fixtures/
19 state = :find_test_case
20 when /class (\w) < Test::Unit::TestCase$/
21 test_case = $1
22 state = :find_test_case
23 end
24
25 when :find_test_case
26 case line
27 when /class (\w) < Test::Unit::TestCase$/
28 test_case = $1
29 state = :find_fixture
30 end
31 end
32 end
33 end
34 end
35 end

Run it like this:


1 $ rake find_missing_fixtures
2 (in D:/wwwroot/wpul.staging.teksol.info)
3 test/unit/name_test.rb: NameTest
4 test/unit/application_helper_test.rb: ApplicationHelperTest
5 test/unit/application_helper_test.rb: ApplicationHelperTruncationTest

Why do I report both the file and TestCase name ? Because I put more than one TestCase per test file, as I reported in Test fixtures and behavioral testing

Of course, if I could do away with fixtures altogether…

See all 3 articles in fixtures

Gem 3 articles

For the first time ever, ragan crystallized why he didn’t like the original #andand implementation, or my own "implementation"http://blog.teksol.info/2007/11/23/a-little-smalltalk-in-ruby-if_nil-and-if_not_nil, #if_not and #if_not_nil and other implementations.

His point is that if you are writing a library, and you want to use the andand gem, you, the author of the library, will pollute the Object namespace of the application that’s using our library.

In effect, the library’s author dependencies will become dependencies of the application that’s using it, even though they don’t want or need the extensions.

The same problem happens with Mongrel and Thin : all the libraries these servers use are loaded in the application that’s running on top of them.

Thank you Reginald for clearing this up for me. I now fully understand the need for the Ruby.rewrite project.

See all 3 articles in gem

Flash 3 articles

I just found about Media Convert, a free online tool that converts files from one format to another. I converted an FLV to SWF, and it worked flawlessly. Their converters also convert from many other formats. Hope this helps someone.

See all 3 articles in flash

Habtm 3 articles

In Rails, there are no requirements that both sides of an association be symmetric. In fact, I often find it useful to have asymmetric relationships.

For example, I’m building a mass-mailer. I have parties (people), emails and recipients. Emails are habtm with regards to parties. Since my join table contains other fields besides the two foreign keys, I need a real model for it.

Examine the following code:


1 class Email < ActiveRecord::Base
2 has_many :recipients
3 end
4
5 class Party < ActiveRecord::Base
6 has_and_belongs_to_many :emails,
7 :join_table => recipients
8 end
9
10 class Recipient < ActiveRecord::Base
11 belongs_to :email
12 belongs_to :party
13
14 def read!
15 self.toggle!(:read)
16 end
17 end

The above models give rise to the following very natural code:


1 >> fbos = Party.find_by_login(fbos)
2 => #<Party:0×3c019d0 …>
3 >> info = Email.create(:subject => Latest Schedule,
4 :body => Group schedule: …)
5 => #<Email:0×3bece38 …>
6 >> info.recipients.create(:party => fbos)
7 => #<Recipient:0×3bccfd8 …>
8 >> fbos.emails
9 => [#<Email:0×3b80dc0 …>]

If access to read or unread E-Mail is needed, the habtm association can be declared more than once, with conditions that discriminate between the different states:


1 class Party < ActiveRecord::Base
2 has_and_belongs_to_many :unread_emails,
3 :join_table => recipients,
4 :conditions => read = 0
5 has_and_belongs_to_many :read_emails,
6 :join_table => recipients,
7 :conditions => read = 1
8 end

Please comment or send me an E-Mail at francois.beausoleil@gmail.com to tell me about your experiences with asymmetric relationships.

See all 3 articles in habtm

Obsolete 3 articles

It hurts, but this article is now obsolete. Rails 2.3 has templates, there are a variety of starter applications, and the text of this article was reformatted and put into Deploying Rails Applications. Good bye, article.

Pinkatio – our latest Rails application

We just had a killer idea to get us thousands of dollars of revenue per month. We will call it pinkatio, and we will put it on Rails.

Creating the Subversion repository

This is the most variable step in the whole procedure. If you know your way around a Unix/Linux system, go ahead and put the repository in /var. Watch your permissions, though. Else, I suggest sticking with a HOME based repository, like I did below. You can use the file repository access method. The only caveat is that you will not be able to share your repository with other people using that method.

To create the repository, we simply call svnadmin’s create subcommand:


1 $ mkdir ~/svn
2 $ svnadmin create —fs-type=fsfs ~/svn/pinkatio

To ease our poor fingers, let us create an environment variable to refer tot he repository’s root URL:


1 $ REPOS=file://`pwd`/svn/pinkatio

Subversion recommends creating all repositories with three folders at the root: trunk, tags and branches. This is if you use the one project per repository. This is explained in more details in Choosing a Repository Layout.

This is by no means a requirement to use Subversion, but I suggest sticking to this convention anyway. Most Subversion repositories I have seen adhere to the convention, and if you have only one project in your repository, it makes sense to be able to tag and branch at will.


1 $ svn mkdir —message="Initial project layout" $REPOS/trunk $REPOS/tags $REPOS/branches
2
3 Committed revision 1.

With the repository creation out of the way, let us now turn to creating our Rails application.

Creating the Rails application and importing into the repository

Creating the Rails application is straightforward:


1 $ rails ~/pinkatio
2 create
3 create app/controllers
4
5 $ cd ~/pinkatio

At this point, you could do an svn import and import the whole application into Subversion. I recommend against doing that. If you use the “in-place import” procedure, you can commit only the pieces that you want, not the whole tree (log files being ones we don’t want under version control). See Subversion’s How can I do an in-place ‘import’ FAQ for the full details.


1 $ svn checkout $REPOS/trunk .
2 Checked out revision 1.

Next, let us add the whole tree to the working copy. This is no different than if we had done an svn import initially, except all changes are local, and we can selectively revert files and folders.


1 $ svn add —force .
2 A app
3
4 A README

The Rails command helpfully creates most of the tree. Since I use migrations in all of my Rails projects, I immediately create the db/migrate/ folder. Edge Rails and Rails 1.1 also include a tmp/ folder. For completeness’ sake, I create it at the same time.


1 $ svn mkdir db/migrate tmp
2 A db/migrate
3 A tmp

Removing the log files from version control

Right now, Subversion will helpfully track changes to the log files. This is not really useful for us, as the log files can be pruned at any point.

To ease our burden, the easiest thing is to tell Subversion to ignore the logs.


1 $ svn revert log/*
2 Reverted ‘log/development.log’
3 Reverted ‘log/production.log’
4 Reverted ‘log/server.log’
5 Reverted ‘log/test.log’
6
7 $ svn propset svn:ignore "*.log" log
8 property ‘svn:ignore’ set on ‘log’

See svn:ignore in the Subversion book for more details on the property format.

Managing the database configuration

Again the Subversion FAQ comes to the rescue: I have a file in my project that every developer must change, but I don’t want those local mods to ever be committed. How can I make ‘svn commit’ ignore the file?.

The solution is to have a template of the file in the repository, and to force each working copy to copy the template file to the real file. Let us simply revert the add of the config/database.yml file, and add a sample file instead:


1 $ svn revert config/database.yml
2 Reverted ‘config/database.yml’
3
4 $ mv config/database.yml config/database.yml.sample
5 $ svn add config/database.yml.sample
6 A config/database.yml.sample
7
8 $ svn propset svn:ignore "database.yml" config
9 property ‘svn:ignore’ set on ‘config’
10 $ cp config/database.yml.sample config/database.yml

The only problem with this procedure is if important changes are made to the config.yml.sample file, the developers might not notice the changes. Most of the time though, the sample file will not change, and leaving it as-is is ok.

Database structure dumps during testing

When you run the tests, Rails dumps the development database’s structure to a file in db/. Usually, this file should not be under version control. Your migration scripts should be under version control instead, and your migrations should enable you to recreate the development database at will.

Additionally, this step will depend on which configuration setting you use for the config.active_record.schema_format. If you use the :ruby (the default on Edge Rails and Rails 1.1), you should ignore the schema.rb file from db/. If you use :sql, simply ignore development_structure.sql instead. Alternatively, you could ignore both files, making this a moot point.


1 $ svn propset svn:ignore "schema.rb" db
2 property ‘svn:ignore’ set on ‘db’

tmp/, documentation, scripts and public

Edge Rails and Rails 1.1 now possess a tmp/ folder. Since this folder will hold socket and session files, we can safely ignore everything in it.


1 $ svn propset svn:ignore "*" tmp
2 property ‘svn:ignore’ set on ‘tmp’

The doc/ folder can hold two subfolders: appdoc/ and apidoc/. If you don’t plan on building the documentation for your project, you can ignore setting svn:ignore on doc/. Else, you should ignore like this:


1 $ svn propset svn:ignore "*doc" doc
2 property ‘svn:ignore’ set on ‘doc’

Subversion also has a property that tells it which files are executable. We can set the property on files that are intended to be run from the command line:


1 $ svn propset svn:executable "" `find script -type f | grep -v ‘.svn’` public/dispatch.
2 property ‘svn:executable’ set on ‘script/performance/benchmarker’
3

Last but not least, my projects usually have a default home page served by a Rails action. This means building a route and removing public/index.html:


1 $ svn revert public/index.html
2 Reverted ‘public/index.html’
3
4 $ rm public/index.html

Saving our work

After all of these changes, it is important to commit our work to the repository.


1 $ svn commit —message="New Rails project"
2 Adding README
3
4 Adding vendor/plugins
5 Transmitting file data …………………………………
6 Committed revision 2.

After this step, it is time to start coding your application, unless you need to go on the Edge…

Using Rails Edge and protecting against overzealous gem upgrades

When you are going to put your application into production, you don’t want an upgrade in your host’s environment to affect your application. To prevent such problems, you should keep a local copy of Rails in your application’s vendor folder.

If you want to live on the Edge (with all the latest features), this step is a necessity. If you are not so comfortable with Edge, replace trunk/ with tags/rel_1-0-0 (or tags/rel_1-1-0 when Rails 1.1 is out) in the svn:externals property below.


1 $ svn propset svn:externals "rails http://dev.rubyonrails.org/svn/rails/trunk/" vendor
2 property ‘svn:externals’ set on ‘vendor’
3
4 $ svn update vendor
5
6 Fetching external item into ‘vendor/rails’
7 A vendor/rails/cleanlogs.sh
8
9 U vendor/rails
10 Updated external to revision 3830.
11
12 Updated to revision 2.

If you went for Rails Edge, you should really rerun the rails . command after you update. This will ensure you get the latest version of the scripts and javascript files. And since we have not made any changes to the contents of any files, now is the best time to do this.


1 $ yes | rails .
2 exists
3 exists app/controllers
4
5 identical log/test.log

Don’t forget to commit your changes:


1 $ svn commit —message="Living on the Edge – set svn:externals on vendor/ for Rails"
2 Sending vendor
3
4 Committed revision 3.

Tracking Edge Rails

Next time you svn update, Subversion will go out to the Rails repository and retrieve all changes since your last checkout. If the JavaScript files changed, you should copy them over using the rails:update Rake command:


1 $ svn update
2
3 Updated external to revision 3831.
4
5 Updated to revision 2.
6
7 $ rake rails:update
8 (in /home/fbos/pinkachio)
9
10 $ svn status
11 M public/javascripts/prototype.js
12 M public/javascripts/effects.js
13 M public/javascripts/dragdrop.js
14 M public/javascripts/controls.js
15
16 $ svn commit —message="Updated JavaScripts to latest revision"
17 M public/javascripts/prototype.js
18

Gems

Gem features a useful subcommand: unpack. When you run it on a gem, it will unpack that gem’s content into the current folder. We can then move the code to our vendor/ folder, and again protect ourselves against host upgrades.

As an example, let us unpack the Money gem:


1 $ cd vendor
2 $ gem unpack money
3 Unpacked gem: ‘money-1.7.1’

Gems all have a lib/ folder into which the gem’s source code is stored. Copying the contents of the lib/ folder into vendor/ is the important trick here. If you move lib/ to vendor, it won’t help, as Rails automatic dependency loading mechanism will not know how to find your code.

At the same time, to comply with the library’s license, we copy the library verbatim to our vendor/ folder.


1 $ cp -Rf money-1.7.1/lib/* .
2 $ cp -Rf money-1.7.1/MIT-LICENSE LICENSE-money
3 $ cp -Rf money-1.7.1/README README-money

Let us tell Subversion what files we now want to version control:


1 $ svn add bank money support money.rb LICENSE-money README-money
2 A bank
3 ..
4 A README-money

To help me remember which version I unpacked, I set a custom property on the main file of the library I just unpacked:


1 $ svn propset version "1.7.1 (Gem)" money.rb
2 property ‘version’ set on ‘money.rb’

Next, we cleanup after ourselves:


1 $ rm -Rf money-1.7.1

Finally, let us commit our changes back to the repository.


1 $ cd ..
2 $ svn commit —message="Unpacked Money 1.7.1 into vendor/"
3 Adding vendor/LICENSE-money
4
5 Transmitting file data ……..
6 Committed revision 4.

Gem upgrades

When the next version of the Money gem will come around, we simply follow the same procedure as above. Of course, you will use svn status to know what files changed exactly. You might have to add new files, and remove old ones.

One tool that can help automate this process svn_load_dirs.pl, from the Subversion’s contrib/ area.

Plugins

For plugins, you have to take the same decision as for Rails – Edge or safe. For plugins, I have found that sticking to released version is safer for me. YMMV.

As an example, I will use the FileColumn plugin. Unfortunately, this plugin is not ready to be used by the script/plugin install -x procedure. So, we have to resort to a manual one.


1 $ svn propset svn:externals "file_column http://opensvn.csie.org/rails_file_column/plugins/file_column/tags/rel_0-3-1/" vendor/plugins
2 property ‘svn:externals’ set on ‘vendor/plugins’
3
4 $ svn update vendor/plugins
5
6 Fetching external item into ‘vendor/plugins/file_column’
7 A vendor/plugins/file_column/test
8
9 U vendor/plugins/file_column
10 Updated external to revision 58.
11
12 Updated to revision 4.

Again, we must not forget to commit our changes to the repository:


1 $ svn commit —message="Added FileColumn plugin"
2 Sending vendor/plugins
3
4 Committed revision 5.

Creating migrations, models and controllers

The Rails generate has a helpful option: --svn (-c):


1 $ script/generate migration —svn InitialSchema
2 exists db/migrate
3 create db/migrate/001_initial_schema.rb
4 A db/migrate/001_initial_schema.rb
5
6 $ svn status
7 A db/migrate/001_initial_schema.rb

All generate and destroy generators accept the --svn option. This makes it easy for the developer to keep his changes under version control.

Final words

I follow this script more or less verbatim for all of my Rails projects. After the first two or three times, this becomes automatic. For the adventurous, I have a shell script which does most of the steps above automatically. You can get rails2.sh.

rails2 license:

# Distributed in the Public Domain by Francois Beausoleil.
# This script might destroy twenty years worth of work, and I cannot be held
# responsible.  You are your own master.  Read this file in detail before
# you use it.
#
# NO IMPLIED WARRANTY.

See all 3 articles in obsolete

User-interface 3 articles

In UI for belongs_to relationships, I discussed a way to make a belongs_to UI. Unfortunately, it was very complex, and I found a better way since then.

I have to thank Michael C. Toren for the spark that inspired this entry.

The way I now do my belongs_to UIs today is much simpler. Let us assume the following models for discussion:

app/models/product.rb

1 class Product
2 belongs_to :color
3
4 validates_presence_of :color
5 end

app/models/color.rb

1 class Color
2 def self.all
3 self.find(:all, :order => name)
4 end
5 end

When creating a product, we would like to select the color of said product. The UI should be a simple SELECT box.

Rails makes it easy to have SELECT boxes coming from some table:

app/views/products/_form.rhtml

1 <p><label>Description:
2 <%= text_field ‘product’, ‘description’ ></p>
3 <p><label>Color:
4 <= collection_select ‘product’, ‘color_id’,
5 Color.all, :id, :name %>
</p>

See how I call #collection_select with color_id instead of color ? That’s the trick to use.

One nice side-effect of this is that validation will still run. The validates_presence_of declaration will be respected if no color_id is assigned.

The only problem left to resolve is the field highlighting that Rails field helpers automagically add when a field validation rule is not respected.

There are two ways to resolve that:

  1. Change the validation to require both color and color_id (adding two messages to the message area), or;
  2. Add the error DIV manually when an error is found on the counterpart field, such as this:




    app/views/products/_form.rhtml

    1 <p><label>Description:
    2 <%= text_field ‘product’, ‘description’ ></p>
    3 <p><label>Color:
    4 <div class="<= ‘fieldWithErrors’ if error_message_on ‘product’, ‘color’ >">
    5 <= collection_select ‘product’, ‘color_id’,
    6 Color.all, :id, :name %>
    </div></p>

UPDATE (2006-01-04): John Indra noted that there didn’t exist an error_messages_on, only error_message_on. Code above corrected.

See all 3 articles in user-interface

General 3 articles

A couple of days ago, @jfcouture tweeted:

@cmercier People with stuff older than 6 months in the Rails world on their blog should do the same!

@jfcouture via Twitter

This was in response to Carl’s rant that obsolete Merb tutorials should be removed / deprecated.

I decided to follow their advice. I’m revisiting all my old posts and saying they are either obsolete, or providing links to replacements. Expect to see at least a post a day for the next couple of weeks as I revisit my old posts. This is interesting! I had forgotten about a lot of things I had written about.

See all 3 articles in general

Mongo 3 articles

Damien Caselli, one of the first brave souls to use Mongo Explorer, reported an issue with time travelling in Mongo Explorer. Luckily, it wasn’t anything serious: just a misunderstanding between me, Cocoa and JavaScript.

To celebrate the No Time Travelling Issues, I’ve released version 1.0.2 of Mongo Explorer. Get yours: mongo-explorer-v1.0.2.zip

Yes, I’m aware of Sparkle, and yes, it will eventually make it into Mongo Explorer. That, and a million other little things.

See all 3 articles in mongo

Shoulda 2 articles

Thoughtbot’s Shoulda is a very nice piece of work. Their ActiveRecord macros are also a god-send:


1 should_protect_attributes :family_id, :debit_account_id, :credit_account_id, :created_at, :updated_at

This code will assert that ActiveRecord attributes are somehow protected (either though attr_accessible or attr_protected). But what about the reverse? There isn’t a macro to do that. I happened to need it, so I implemented it on my own fork of Shoulda.

This allows us to specify:


1 should_protect_attributes :family_id, :debit_account_id, :credit_account_id, :created_at, :updated_at
2 should_allow_attributes :family, :debit_account, :credit_account, :amount, :posted_on, :description

NOTE: Example taken from my family budget application.

See all 2 articles in shoulda

Routing 2 articles

Rails allows you to define routes with named prefixes. I expected Rails to somehow know that I wanted my prefix to separate the rest of the name with an underscore.

Here was my original routes:

config/routes.rb

1 ActionController::Routing::Routes.draw do |map|
2 map.resources :parties, :path_prefix => "/admin" do |parties|
3 parties.resources :addresses,
4 :controller => "address_contact_routes",
5 :name_prefix => :party
6 end
7
8 map.resources :addresses,
9 :controller => "address_contact_routes"
10 end

Then, when I tried to use my route, I was getting a NoMethodError:


1 1) Error:
2 test_can_show(AddressContactRoutesControllerTest::PartyWithAddressTest):
3 ActionView::TemplateError: undefined method `party_address_path’ for #<#<Class:0xb768f6ac>:0xb63ee41c>
4 On line #3 of app/views/address_contact_routes/address_contact_route.rhtml
5
6 1: <%
7 2: if address_contact_route.routable && !address_contact_route.new_record? then
8 3: update_url = "#{party_address_path(address_contact_route.routable, address_contact_route)}.txt"
9 4: end
10 5: -%>
11 6: <% inline_fields_for(:address, address_contact_route, :url => update_url) do |f| -%>
12
13 #{RAILS_ROOT}/app/views/address_contact_routes/address_contact_route.rhtml:3:in `_run_rhtml_47app47views47address_contact_routes47_addresscontact
route46rhtml’

After a bit of sleuthing (and adding messages to ActionController::Routing::RouteSet#add_named_route), I found out that I was supposed to put the underscore myself on the prefix, like this:


1 map.resources :parties, :path_prefix => "/admin" do |parties|
2 parties.resources :addresses,
3 :controller => "address_contact_routes",
4 :name_prefix => :party_
5 end

It was also interesting to see all those generated routes. I discovered that the route with a format was named formatted_whatever.

Here’s a diff against the 1.2 branch of Rails that allows you to see all the generated routes as they are read:


1 $ svn diff vendor/rails/actionpack/lib/action_controller
2 Index: vendor/rails/actionpack/lib/action_controller/routing.rb
3 ===============
4 - vendor/rails/actionpack/lib/action_controller/routing.rb (revision 6424)
5 + vendor/rails/actionpack/lib/action_controller/routing.rb (working copy)
6 @ -349,7 +349,9 @
7
8 method_decl = "def generate_extras(#{args})\npath, hash = generate_raw(options, hash, expire_on)\n[path, extra_keys(options)]\nend"
9 instance_eval method_decl, "generated code (#{__FILE__}:#{__LINE__})"
10 – raw_method
11 + returning raw_method do
12 + RAILS_DEFAULT_LOGGER.debug {raw_method}
13 + end
14 end
15
16 # Build several lines of code that extract values from the options hash. If any
17 @ -999,6 +1001,7 @
18 deprecate :root => "(as the the label for a named route) will become a shortcut for map.connect ’’, so find another name"
19
20 def method_missing(route_name, *args, &proc)
21 + RAILS_DEFAULT_LOGGER.debug {"ActionController::Routing::RouteSet::Mapper#method_missing(#{route_name.inspect}, #{args.inspect})"}
22 super unless args.length >= 1 && proc.nil?
23 set.add_named_route(route_name, *args) <span class="no">24</span> end <span class="no"><strong>25</strong></span> @ -1175,6 +1178,7 @@
26 end
27
28 def add_named_route(name, path, options = {})
29 + RAILS_DEFAULT_LOGGER.debug {"ActionController::Routing::RouteSet#add_named_route(#{name.inspect}, #{path.inspect}, #{options.inspect})"}
30 named_routes[name] = add_route(path, options)
31 end
32

For reference, here are all the generated routes for this declaration:

config/routes.rb

1 ActionController::Routing::Routes.draw do |map|
2 map.resources :layouts, :path_prefix => "/admin"
3 end

layouts =&gt; "/admin/layouts"
formatted_layouts =&gt; "/admin/layouts.:format"
new_layout =&gt; "/admin/layouts/new"
formatted_new_layout =&gt; "/admin/layouts/new.:format"
edit_layout =&gt; "/admin/layouts/:id;edit"
formatted_edit_layout =&gt; "/admin/layouts/:id.:format;edit"
layout =&gt; "/admin/layouts/:id"
formatted_layout =&gt; "/admin/layouts/:id.:format"

EDIT (2007-03-14 16:52 EDT): Changed link to routing.rb file to the Rails Trac Browser.

See all 2 articles in routing

Gutsy gibbon 2 articles

I just posted about my problem with the English US keyboard not having a pipe character ? Well, I removed the French Canadian keyboard layout from my configuration, and now I have my pipe character back. That’s not very nice, since I won’t be able to type in French anymore, but at least, I can program again.

Thanks to dvorak keyboard layout but keyboard shortcuts are in qwrty for the idea.

See all 2 articles in gutsy gibbon

Documentation 2 articles

A friend of mine asked me a question today. He wanted to know “When are associated objects saved” ?

See ActiveRecord::Associations. Look for the section titled “Unsaved objects and associations”.

See all 2 articles in documentation

Defensio 2 articles

Why should you switch to Defensio instead of Akismet ? Because of this:

The Mephisto interface, when interfaced with Defensio shows the spaminess of each comment, as well as use different shades of orange to show spammy comments
Click for larger version

Defensio provides useful statistics to the blogger. A typical response would look like:


1 —-
2 defensio-result:
3 message: ""
4 status: success
5 signature: awnc057e1a132p1jj3t4x
6 spaminess: 0.7
7 api-version: "1.1"
8 spam: true

Notice how Defensio returned the spaminess and a signature ? The signature can be used to retrain Defensio. When a false positive or negative comes through, the Defensio API simply accepts a series of signatures, and will retrain itself. Since the signature is a short-hand for the whole comment, all of the data is available for retraining: IP, author email, author name, comment’s body, etc.

It is not obvious from the screenshot above (as I mostly get spammy comments), but there are actually many shades of orange to highlight the spaminess of comments. Lighter shades are non-spammy, and darker ones, spammier.

See all 2 articles in defensio

Datamapper 2 articles

In Part I, I showed how to configure Rails to use DataMapper instead of ActiveRecord. Well, after some thought, I realized we couldn’t really test anything. Here’s the proof:


1 class ArticleTest < ActiveSupport::TestCase
2 def test_database_is_empty
3 assert Article.all.empty?
4 end
5 end

Running the tests results in a failure:


1 $ rake test:units
2 (in /Users/francois/Documents/work/dm_on_rails)
3 /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test "/Library/Ruby/Gems/1.8/gems/rake-0.8.2/lib/rake/rake_test_loader.rb" "test/unit/article_test.rb"
4 Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.2/lib/rake/rake_test_loader
5 Started
6 .F
7 Finished in 0.049976 seconds.
8
9 1) Failure:
10 test_database_is_empty(ArticleTest)
11 [./test/unit/article_test.rb:10:in `test_db_empty_on_start’
12 /Library/Ruby/Gems/1.8/gems/activesupport-2.1.0/lib/active_support/testing/setup_and_teardown.rb:67:in `__send__’
13 /Library/Ruby/Gems/1.8/gems/activesupport-2.1.0/lib/active_support/testing/setup_and_teardown.rb:67:in `run’]:
14 <false> is not true.
15
16 2 tests, 2 assertions, 1 failures, 0 errors
17 rake aborted!
18 Command failed with status (1): [/System/Library/Frameworks/Ruby.framework/…]
19
20 (See full trace by running task with —trace)

Time to open the wonderful world of transactions! At the time of this writing, DataMapper is at 0.9.6. There is a comment in lib/dm-core/transaction.rb saying TODO: move to dm-more/dm-transactions. If the code below doesn’t work, add a new dependency on dm-transaction.

So, let’s begin by adding new setup / teardown hooks to Test::Unit::TestCase:

test/test_helper.rb

1 class Test::Unit::TestCase
2 setup do
3 __transaction</span> = <span class="co">DataMapper</span>::<span class="co">Transaction</span>.new(<span class="co">DataMapper</span>.repository(<span class="sy">:default</span>)) <span class="no"> 4</span> <span class="iv">transaction.begin
5
6 # FIXME: Should I really be calling #push_transaction like that, or is there a better way?
7 DataMapper.repository(:default).adapter.push_transaction(@
transaction)
8 end
9
10 teardown do
11 if __transaction</span> <span class="no">12</span> <span class="co">DataMapper</span>.repository(<span class="sy">:default</span>).adapter.pop_transaction <span class="no">13</span> <span class="iv">transaction.rollback
14 @
transaction = nil
15 end
16 end
17 end

With that code in place, we can run the tests again and see the results of our hard work:


1 $ rake test:units
2 (in /Users/francois/Documents/work/dm_on_rails)
3 /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test "/Library/Ruby/Gems/1.8/gems/rake-0.8.2/lib/rake/rake_test_loader.rb" "test/unit/article_test.rb"
4 Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.2/lib/rake/rake_test_loader
5 Started
6 ..
7 Finished in 0.009485 seconds.
8
9 2 tests, 2 assertions, 0 failures, 0 errors

See all 2 articles in datamapper

S3 2 articles

Today sees a new release of Cliaws. This is a simple library to interact with S3 (and other services) through the command-line. This library is very similar to Amazon’s own ec2-* scripts, except this library is written in Ruby.

Interacting with S3 should be no harder than:


1 $ clis3 put my-local-file my-bucket/my-s3-file

Put many files at once:


1 $ clis3 put my-local-file0 my-local-file1 my-bucket/my-s3-directory/

Put an environment variable:


1 $ clis3 put —data $MY_DATA my-bucket/my-env-value

Put stdin too!


1 $ tar czfv /var/cache/mylvmbackup/backup | clis3 put – my-bucket/my-backup/backup-20080825-000000.tar.gz

All of this functionnality is available from with Ruby too:


1 require "rubygems"
2 require "cliaws"
3
4 Cliaws.s3.put("this is the content", "my-bucket/my-s3-file")
5 File.open("my-local-file", "rb") do |io|
6 Cliaws.s3.put(io, "my-bucket/my-s3-file")
7 end

Give the RubyForge gem servers a couple of hours to refresh themselves, and then enjoy!

See all 2 articles in s3

Sqs 2 articles

Starting from a fresh Rails application (I’m using 2.0.2), install AttachmentFu:


1 script/plugin install http://svn.techno-weenie.net/projects/plugins/attachment_fu/

Edit config/amazon_s3.yml and put this:

config/amazon_s3.yml

1 development:
2 bucket_name: amazon-sqs-development-yourname
3 access_key_id: "your key"
4 secret_access_key: "your secret access key"
5 queue_name: amazon-sqs-development-resizer-yourname

queue_name is new. AttachmentFu does not require this, but we are going to reuse the file from our own code, so better put all configuration in the same place.

Generate a scaffolded Photo model using:


1 $ script/generate scaffold photo filename:string size:integer content_type:string width:integer height:integer parent_id:integer thumbnail:string

Edit app/views/photos/new.erb.html and replace everything with this:

app/views/photos/new.erb.html

1 <h1>New photo</h1>
2
3 <%= error_messages_for :photo >
4
5 < form_for(@photo, :html => {:multipart => true}) do |f| >
6 <p>
7 <label for="photo_uploaded_data">File:</label>
8 <= f.file_field :uploaded_data >
9 </p>
10
11 <p>
12 <= f.submit "Create" >
13 </p>
14 < end >
15
16 <= link_to ‘Back’, photos_path %>


What we did here is simply tell Rails to use a multipart encoded form, and to only provide us with a single file upload field.

Edit app/models/photo.rb and add the AttachmentFu plugin configuration:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 has_attachment :content_type => :image, :storage => :s3
3 validates_as_attachment
4 end

Start your server and confirm you can upload a file. No thumbnails were generated as we did not configure any thumbnailing to do. We don’t actually want AttachmentFu to handle that, so we can’t just specify it in the has_attachment call.

To use RightScale’s AWS SQS component, we have to configure it with the access key and secret access key. Add this to the end of the Photo class:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def queue
3 self.class.queue
4 end
5
6 class << self
7 def queue
8 # This creates the queue if it doesn’t exist
9 queue</span> ||= sqs.queue(aws_config[<span class="s"><span class="dl">&quot;</span><span class="k">queue_name</span><span class="dl">&quot;</span></span>]) <span class="no"><strong>10</strong></span> <span class="r">end</span> <span class="no">11</span> <span class="no">12</span> <span class="r">def</span> <span class="fu">sqs</span> <span class="no">13</span> <span class="iv">sqs ||= RightAws::Sqs.new(
14 aws_config["access_key_id"], aws_config["secret_access_key"],
15 :logger => logger)
16 end
17
18 def aws_config
19 return aws_config</span> <span class="r">if</span> <span class="iv">aws_config
20
21 aws_config</span> = <span class="co">YAML</span>.load(<span class="co">File</span>.read(<span class="co">File</span>.join(<span class="co">RAILS_ROOT</span>, <span class="s"><span class="dl">&quot;</span><span class="k">config</span><span class="dl">&quot;</span></span>, <span class="s"><span class="dl">&quot;</span><span class="k">amazon_s3.yml</span><span class="dl">&quot;</span></span>))) <span class="no">22</span> <span class="iv">aws_config = aws_config</span>[<span class="co">RAILS_ENV</span>] <span class="no">23</span> raise <span class="co">ArgumentError</span>, <span class="s"><span class="dl">&quot;</span><span class="k">Missing </span><span class="il"><span class="idl">#{</span><span class="co">RAILS_ENV</span><span class="idl">}</span></span><span class="k"> configuration from config/amazon_s3.yml file.</span><span class="dl">&quot;</span></span> <span class="r">if</span> <span class="iv">aws_config.nil?
24 @aws_config
25 end
26 end
27 end

#aws_config is a method that reads the configuration. #sqs is a method that provides access to an instance of RightScale::Sqs, pre-configured with the correct access keys. #queue uses #sqs to get or create a named queue. There’s also an instance version of #queue, to ease our code later on.

Let’s add the request sending:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def send_resize_request
3 # Don’t send a resize request for thumbnails
4 return true unless self.parent_id.blank?
5
6 params = Hash.new
7 params[:id] = self.id
8 params[:sizes] = Hash.new
9 params[:sizes][:square] = "75×75"
10 params[:sizes][:thumbnail] = "100x"
11
12 begin
13 queue.push(params.to_yaml)
14 rescue
15 logger.warn {"Unable to send resize request. Error: #{$!.message}"}
16 logger.warn {$!.backtrace.join("\n")}
17
18 # Don’t raise the error so the request goes through.
19 # We don’t want the user to see a 500 error because
20 # we can’t talk to Amazon.
21 end
22 end
23 end

Now, this is getting interesting. AttachmentFu knows if the current model is a thumbnail or not by looking at parent_id. If it’s nil, we are the parent, else we are a thumbnail. We do the same thing here.

Then, we setup a couple of parameters to send to the resizer. Notice we send the actual thumbnail sizes in the message itself.

Next, we do the most important part: queue.push. This sends a message string (limited to 256 KiB) to Amazon SQS, and returns. If there is an error, we don’t actually want to prevent the request from completing, so we rescue any exceptions and log them. If you have the ExceptionNotifier plugin installed, this is a good place to log to it.

Now that we have a way to send the resize request, we have to execute it at some point. The controller is not the right place to do it. If you create Photo models from more than one controller, you’re bound to forget to call #send_resize_request. It’s better to do it in an #after_create callback, which we’ll do with a single line:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 after_create :send_resize_request
3 end

Next, we have to receive the messages. So, we write a new method in Photo:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 class << self
3 def fetch_and_thumbnail
4 messages = queue.receive_messages(20)
5 return if messages.blank?
6
7 logger.debug {"==> Photo\#fetch_and_thumbnail — received #{messages.size} messages"}
8 messages.each do |message|
9 params = YAML.load(message.body)
10 photo = Photo.find_by_id(params[:id])
11 if photo.blank? then
12 # The Photo was deleted before we got a chance to thumbnail it.
13 # We must delete the message, or we’ll always get it afterwards.
14 message.delete
15 next
16 end
17
18 photo.generate_thumbnails(params[:sizes])
19 message.delete
20 end
21 end
22 end
23 end

The first thing we do is see if there are any messages. The call to #queue is the helper method we defined earlier on. We ask to receive up to 20 messages at a time. If there were no messages, we simply return.

Then, for each message, we have to process it, so we iterate over each message, retrieving the original parameters Hash. The important thing to do is to delete the message after we have processed it, or else the message will still be visible next time around.

#generate_thumbnails is important, but uninteresting in this discussion.

See all 2 articles in sqs

Funny 2 articles

How’s that for being helpful?


1 $ rake db:migrate —trace
2 (in /Users/francois/Documents/work/family_budget)
3 * Invoke db:test:prepare (first_time)
4 *
Invoke environment (first_time)
5 * Execute environment
6 *
Invoke db:abort_if_pending_migrations (first_time)
7 * Invoke environment
8 *
Execute db:abort_if_pending_migrations
9 You have 13 pending migrations:
10 1 CreateFamilies
11 2 CreatePeople
12 3 CreateAccounts
13 4 CreateTransfers
14 5 CreateTransferMembers
15 6 AddFamilyToPeople
16 7 DestroyTransferMember
17 8 AddDebitAccountIdCreditAccountIdAndAmountToTransfers
18 9 CreateBudgets
19 10 AddAdminFlagToPerson
20 11 DefaultBudgetAmountToZeroFromNull
21 20081103020010 CreateTransactions
22 20081103020142 CreateBankAccounts
23 Run "rake db:migrate" to update your database then try again.

Hmm, I’m trying to run migrations, stupid…

I finally traced the problem (using git bisect !) to this:

lib/tasks/cucumber.rake

1 $:.unshift(RAILS_ROOT + /vendor/plugins/cucumber/lib)
2 require cucumber/rake/task
3
4 namespace :test do
5 desc "Runs the Cucumber features tests (integration tests)"
6 Cucumber::Rake::Task.new(:features) do |t|
7 ENV["RAILS_ENV"] = RAILS_ENV = "test"
8 Rake::Task["db:test:prepare"].invoke
9 Rake::Task["db:fixtures:load"].invoke
10 t.cucumber_opts = "—format pretty"
11 end
12 end

Can you spot the error? The problem originates with setting RAILS_ENV in this code block. But isn’t the block supposed to be executed later? When the block is #call’ed? Very strange. At the moment, I simply commented out the line, but that’s going to bite me in a couple moments when I’m ready to test my features again… Oh joy of debugging!

See all 2 articles in funny

Functional-testing 2 articles

This is the test I wrote. You’ll notice it is almost verbatim the scaffolded test.


1 def test_create
2 num_blog_accounts = BlogAccount.count
3
4 post :new,
5 :blog_account => {:feed_url => http://bladibla.com/},
6 :commit => Close
7 assert_redirected_to blog_accounts_url
8
9 assert_equal num_blog_accounts + 1, BlogAccount.count
10 end

Interestingly, I made it fail with the following simple code:


1 def new
2 return edit unless params[:id].blank?
3 title</span> = <span class="s"><span class="dl">'</span><span class="k">Creating blog account</span><span class="dl">'</span></span> <span class="no"> 4</span> <span class="iv">blog_account = BlogAccount.new
5 save_and_navigate if request.post?
6 end
7
8 protected
9 def save_and_navigate
10 if @blog_account.update_attributes(params[:blog_account])
11 flash_success BlogAccount was successfully updated.
12 respond_to do |wants|
13 wants.js do
14 render :action => save, :content_type => text/javascript; charset=utf-8, :layout => false
15 end
16 wants.html do
17 redirect_to blog_accounts_url
18 end
19 end
20 end
21 end

The failure was:


1 $ ruby test\functional\admin\blog_accounts_controller_test.rb -n test_create
2 Loaded suite test/functional/admin/blog_accounts_controller_test
3 Started
4 F
5 Finished in 0.188 seconds.
6
7 1) Failure:
8 test_create(Admin::BlogAccountsControllerTest) [test/functional/admin/blog_accounts_controller_test.rb:42]:
9 Expected response to be a <:redirect>, but was <200>
10
11 1 tests, 1 assertions, 1 failures, 0 errors

Flipping the order of my respond_to block did the trick:


1 respond_to do |wants|
2 wants.html do
3 redirect_to blog_accounts_url
4 end
5 wants.js do
6 render :action => save, :content_type => text/javascript; charset=utf-8, :layout => false
7 end
8 end

This is partially logical. Of course, the request doesn’t have an Accept header, so Rails uses the default, which is to return in order the order of definition.

Alternatively, setting the request’s Accept header would do the trick:


1 def setup
2 controller</span> = <span class="co">Admin</span>::<span class="co">BlogAccountsController</span>.new <span class="no">3</span> <span class="iv">request = ActionController::TestRequest.new
4 response</span> = <span class="co">ActionController</span>::<span class="co">TestResponse</span>.new <span class="no"><strong>5</strong></span> <span class="no">6</span> <span class="iv">request.env[HTTP_ACCEPT] = text/html
7 end

See all 2 articles in functional-testing

Capistrano 2 articles

If like me you didn’t know, you can do that with Capistrano:


1 $ cap HOSTS=web0.myapp.com deploy:update_code
2 cap ROLES=web,app deploy

Not terribly useful for the tasks demonstrated above, but what about this ?


1 $ cap ROLES=app monit:summary

Incidentally, where are these documented ? I dimly remembered reading this somewhere once, and tried it. Works like a charm!

See all 2 articles in capistrano

Associations 2 articles

A friend of mine asked me a question today. He wanted to know “When are associated objects saved” ?

See ActiveRecord::Associations. Look for the section titled “Unsaved objects and associations”.

See all 2 articles in associations

Control 2 articles

Daniel Berlinger asks a good question about my Sharing models between two Rails applications using Piston asking:

Without having a moment to think about it… there must be other ways to accomplish this (in Rails) without Piston, and other Subversion trickery. No?

Daniel Berlinger

I know of only one way to do it: create a plugin to hold the models, and load this plugin into all of the applications that need the shared models.

See all 2 articles in control

Xlsuite 2 articles

Ouch! That hurts! Thurdsay (yesterday) I finished setting up new DB instances for XLsuite. All was well, and the servers performed admirably well for 24 hours. Then, suddenly, the LVM partition that held /var/lib/mysql went away…

Around 18:08 UTC (11:08:09 PDT), our main MySQL database server went down. Luckily, yesterday (Thurdsay), I had just replaced our whole DB infrastructure to have a replicated master/slave setup. It took us 15 minutes to notice that the sites were down, and another 20 minutes to execute a database failover. By 18:50 UTC (11:50 PDT), things were back to normal.

François Beausoleil

Unfortunately, I did not have the scripts in place yet to do the database failover, so it was a manual process. It wasn’t too hard, but I did have to remember one thing:


1 Mysql::Error: The MySQL server is running with the —read-only option so it cannot execute this statement: INSERT INTO sessions (`updated_at`, `sessid`, `data`) VALUES

In High Performance MySQL, the authors recommend, in Chapter 8:

On the slave, we recommend enabling the following configuration options:

skip_slave_start
read_only

Double oops! Anyway, the advice in the book was invaluable to me. If you manage MySQL and don’t already have the book, I highly suggest you buy it.

See all 2 articles in xlsuite

Library 2 articles

I have a passing interest in Git. I haven’t migrated my projects over to it now, but am looking into it. Just for fun, I googled “ruby git”, and I found Ruby/Git. I thought more people should know about this.

See all 2 articles in library

Cocoa 2 articles

Damien Caselli, one of the first brave souls to use Mongo Explorer, reported an issue with time travelling in Mongo Explorer. Luckily, it wasn’t anything serious: just a misunderstanding between me, Cocoa and JavaScript.

To celebrate the No Time Travelling Issues, I’ve released version 1.0.2 of Mongo Explorer. Get yours: mongo-explorer-v1.0.2.zip

Yes, I’m aware of Sparkle, and yes, it will eventually make it into Mongo Explorer. That, and a million other little things.

See all 2 articles in cocoa

Subversion 2 articles

Now that I’ve had a good change to play with Git, I’m ready to implement Git support in Piston. This will involve a couple of refactorings. I can see 4 cases currently:

Repository Working Copy Strategy
Subversion Subversion Use current strategy of storing data in Subversion properties
Subversion Git Clone the Git repository, copy the files over and store the metadata as Subversion properties. Use Git to handle the merging for Piston (Yay!)
Git Subversion svn export the data and use a hidden YAML file to store the metadata in the pistonized directory
Git Git Use Git submodules perhaps ? Or git clone + copy + YAML

I have no idea how git submodules work, so I can’t really say that I will be handling that last case in the most efficient manner. I’m planning on having completed this work by the end of next week (March 14th). Stay tuned for details !

See all 2 articles in subversion

Generator 2 articles

Engine Generator

2005-12-21

This article is obsolete now that Rails 2.3 has Engines in core.

Jon Lim on the Rails mailing list just announced the release of Engines Generator.

From the documentation:

$ script/generate engine UploadEngine
Please enter the author’s name: Joe Smith
Please enter the author’s email: joe.smith@xyz.com

We can generate the following licenses automatically for you:
0) None
1) GPL
2) MIT
3) LGPL
Please select a license: 2
‘MIT’ selected
exists vendor/plugins
create vendor/plugins/upload_engine
create vendor/plugins/upload_engine/README
create vendor/plugins/upload_engine/init_engine.rb
create vendor/plugins/upload_engine/app
create vendor/plugins/upload_engine/app/models
create vendor/plugins/upload_engine/app/controllers
create vendor/plugins/upload_engine/app/helpers
create vendor/plugins/upload_engine/app/views
create vendor/plugins/upload_engine/db
create vendor/plugins/upload_engine/db/migrate
create vendor/plugins/upload_engine/lib
create vendor/plugins/upload_engine/lib/upload_engine.rb
create vendor/plugins/upload_engine/lib/upload_engine
create vendor/plugins/upload_engine/public
create vendor/plugins/upload_engine/public/javascripts
create vendor/plugins/upload_engine/public/javascripts/upload_engine.js
create vendor/plugins/upload_engine/public/stylesheets
create vendor/plugins/upload_engine/public/stylesheets/upload_engine.css
create vendor/plugins/upload_engine/tasks
create vendor/plugins/upload_engine/tasks/upload_engine.rake
create vendor/plugins/upload_engine/test
create vendor/plugins/upload_engine/test/test_helper.rb
create vendor/plugins/upload_engine/test/fixtures
create vendor/plugins/upload_engine/test/functional
create vendor/plugins/upload_engine/test/unit

$ mv app/models/upload.rb vendor/plugins/upload_engine/app/models
$ mv app/controllers/upload_controller.rb vendor/plugins/upload_engine/app/controllers
$ mv app/views/upload vendor/plugins/upload_engine/app/views
$ mv public/javascripts/upload_engine.js vendor/plugins/upload_engine/public/javascripts
$ mv public/stylesheets/upload_engine.css vendor/plugins/upload_engine/public/stylesheets

config/environment.rb

1 Engines.start :upload_engine

$ script/server

This is way cool. Great work, Jon !

See all 2 articles in generator

Desktop 2 articles

I upgraded to Ubuntu Gutsy Gibbon today. First impressions: not much changes. That’s on the surface, of course. The system came up normally, and I could start my applications.

There’s a new Screen and Graphics Preferences for setting up dual screens. Unfortunately, after 20 minutes I couldn’t get my second screen to come up correctly. I have other things to do, so I stopped playing with that.

On the downside, my keyboard layout moved some keys. I can’t put a pipe character (|) on the command line. That’s very bad for me: no Ruby block parameters, no Smalltalk temporary variable declarations, and certainly no command chaining on the command line.

All in all, I should have waited another month or so before upgrading. I’ll know better next time around. But I still love my Ubuntu Desktop, compared to the klunky Windows XP.

See all 2 articles in desktop

Todoapp 2 articles

Ramon Leon sent me an E-Mail:

Attached are a few more idiomatic changes, the package is commented.

Please see the Monticello repository if you are interested.

See all 2 articles in todoapp

Montreal on rails 2 articles

Yesterday (Tuesday May 20th, 2008), I presented at Montreal on Rails. I made a short and sweet presentation on Mephisto, and how I refactored it to support both Akismet and Defensio.

You can grab the slides for “Refactoring to Patterns: How Mephisto went from a single engine Lada to a multi-engine jet fighter”/2008/05/21/refactoring-to-patterns.pdf (PDF).

References

Design Patterns

Other interesting patterns that I used in Mephisto, which I briefly talked about, but haven’t mentioned in the slides at all:

Refactoring

  • Refactoring to Patterns

Other things I talked about

See all 2 articles in montreal on rails

Productivity 2 articles

If you’re in any way, shape or form like me, you usually have more energy in the morning (if you don’t, skip this post). My regular schedule puts me in my office around 9 AM every weekday. First thing I do is pop open Google Reader, and start reading blogs. What an utter waste of time… I waste perfectly good coding time to do an activity that’s similar to watching television. I can waste my time in the afternoon, when I have less energy. I say waste but I really mean relax. Don’t take me wrong: reading blogs is a very important time, but I should just do it when it’s OK to do, not when I have the most energy.

Yesterday evening, I was pondering on the why of the above. Turns out I can procrastinate just as well as anyone else…

Procrastination is like masturbation… it's good in the beginning, but in the end, you realize you've just fucked yourself

See all 2 articles in productivity

Continuous-integration 2 articles

I’m just setting up Continuous Integration for all of our branches on XLsuite.com. Doing a search for “cruisecontrol.rb” on Google unearthed: Ci For The Web 2.0 Guy Or Gal. I’m running the steps right now on my RimuHosting VPS.

Things are progressing along quite rapidly. I’ll say more when I’m done.

See all 2 articles in continuous-integration

Opensource 2 articles

Would you believe we just GPL’d nearly 40,000 lines of code? Yes we did

See all 2 articles in opensource

Bug 2 articles

This is the initial release of the Windows Tempfile Fix plugin. This plugin sets out to make Tempfile binary safe on Windows.

Installation


1 $ piston import svn://svn.teksol.info/svn/rails/plugins/windows_tempfile_fix vendor\plugins\windows_tempfile_fix

Alternatively:


1 $ ruby script/plugin install svn://svn.teksol.info/svn/rails/plugins/windows_tempfile_fix

README

As I reported on my blog at
http://blog.teksol.info/articles/2006/12/08/rmagick-and-jpeg-datastream-contains-no-image,
the Windows version of Ruby will fail to load certain files when they are used
through a Tempfile. This is because the Tempfile class does not open it’s files
in binary mode.

This plugin is very conservative. It will not install itself if it does
not detect the exact version it is supposed to be used against.

This plugin will only do it’s work if it the following expression evaluates
to true:
RUBY_PLATFORM == “i386-mswin32” && RUBY_VERSION == “1.8.4”

See all 2 articles in bug

Conditions 2 articles

This article is obsolete. You may find it’s replacement at Building the SQL WHERE Clause Dynamically – updated

In Building the SQL WHERE clause dynamically in Rails, I showed how to build the WHERE clause like this:


1 class SearchController < ApplicationController
2 def search
3 conditions = [1=1]
4
5 conditions << cond1 = :cond1 if params[:cond1]
6 conditions << cond2 = :cond2 if params[:cond2]
7
8 @results = Model.find(:all,
9 :conditions => [conditions.join( AND ), params])
10 end
11 end

Ezra Zygmuntowicz of brainsp.at fame created a Cond module to make that even easier:


1 class SearchController < ApplicationController
2 def search
3 conditions = Cond::create do
4 cond1 =, params[:cond1]
5 cond2 =, params[:cond2]
6 end
7
8 @results = Model.find(:all,
9 :conditions => conditions)
10 end
11 end

See Build AR::Base.find’s :conditions clause dynamically take one for the full source code.

Now, if that were turned into a plugin, wouldn’t that be nice ?

See all 2 articles in conditions

Todo 2 articles

Via The Weekly Squeak, I found a link to a new Seaside tutorial. The Software Architecture Group of the Hasso-Platter-Institut implemented a todo application tutorial Seaside.

The tutorial is very complete:

Extern resources like images, css or javascript do belong to a proper website as well. Chapter seven contains the possibilities you have with Seaside and their pros and cons. Up to now, the whole application is only practicable for one time, afterwards all values of the user are forgotten. This problem of the persistence is treated by the eight chapter which, next to it, presents three different possibilities in detail. … In the nineth part the focus is put on an additional library which makes it possible to implement Ajax in Seaside Websites. Script.aculo.us with the integration by Lukas Renggli offers an easy and simple way to create your own Website in the style of Web 2.0.

Go and read it. Very good !

See all 2 articles in todo

Upgrade 2 articles

Just got bitten by a bug. In a model, I was doing:


1 self.unit_price = self.product.current_price(Date.today) \
2 if self.unit_price.cents.blank?

After upgrading to Rails 0.14.3, I found some failures in my tests. About 30 minutes of sleuthing around, and I found my bug to be with the new object.blank? behavior.

Some console code to “prove” it:


1 $ ruby script\console
2 Loading development environment.
3 >> 0.blank?
4 => true
5 >> exit
6
7 $ svn up vendor\rails
8
9 U vendor\rails
10 Updated external to revision 2932.
11
12 $ ruby script\console
13 Loading development environment.
14 >> 0.blank?
15 => false

So, I rewrote my code to now do:


1 self.unit_price = self.product.current_price(Date.today) \
2 if 0 == self.unit_price.cents

Again, this shows how a good set of unit and functional tests can help prevent problems before they hit you.

See all 2 articles in upgrade

Link 2 articles

James Bowes has a super simple git-rebase explanation: git rebase: keeping your branches current.

If you have any difficulty understanding the concept, or want an easy way to explain to someone, go ahead.

I’m just starting to use git-svn and git-rebase, and the combination rocks.

See all 2 articles in link

Miscellaneous 2 articles

I built an application for a non-profit in my region. It allows them to track donations and statistics about their work. One thing that’s very important for them is to know if / when they called a previous donor, to see if they will give again this year. They also need to emit receipts for those people that donated. And they must not emit the same receipt twice, etc. You know the drill. I have a page in the application where they can build such a query:

On this page, they can choose which campaign they’re working on, and different event types that occur during a campaign. They can also choose how to present the result: the date at which the event occured, or the sum of money or quantity they acquired. It’s a large form, since there are almost 15 event types that can occur in a single campaign.

René, my customer, asked me if they could save queries between sessions, so they can go home in the evening, and come back the next day. I said “Sure, I’ll just need a Search model here, and … Hold on a minute: I have a much simpler solution: bookmark the result page.”

You see, I built this application on REST principles. The query itself is idempotent: it either shows events, or not. And since this is idempotent, a GET query is just what the doctor ordered. And browsers are nice enough to allow bookmarking GET requests. Yup: 10 hours of work just disappeared because I used what’s already available. My customer was happy: he didn’t have to wait a minute to get the functionality he needed. I was happy because I had 10 hours of work NOT to do, and I don’t have to maintain that code: it’s in somebody else’s hands. TSTTCPW

This was a real win-win situation.

See all 2 articles in miscellaneous

Jruby 2 articles


1 # JRuby 1.6.0 required
2 require "sequel"
3 require "jdbc/postgresql"
4
5 DB = Sequel.connect "jdbc:postgresql://127.0.0.1:5432/mydb"
6
7 table = DB[:mytable]
8 time = Benchmark.measure do
9 (1..10000).each do |n|
10 table.insert(:a => n, :b => 2*n)
11 end
12 end
13
14 puts time

How much time do you think this is going to take? This is the most inefficient way to insert many rows to a database. Remember each call to #insert will do a round trip to the database. Even if your round trip time is 1ms, you’ll still pay for 10 seconds of round trip time, time which your program could be doing something much more useful, such as generating revenue (somehow).

Instead, you should use the bulk copy feature of your database engine. In my case, that’s PostgreSQL. Since I’m using JRuby, I have to turn to the JDBC world, but that’s all right: everything has been implemented already, by someone, somewhere. I’ll refer you to the relevant pages:

And the relevant code would be:


1 # JRuby 1.6.0 required
2 require "sequel"
3 require "jdbc/postgresql"
4 require "java"
5
6 DB = Sequel.connect "jdbc:postgresql://127.0.0.1:5432/mydb"
7
8 time = Benchmark.measure do
9 DB.synchronize do |connection|
10 copy_manager = org.postgresql.copy.CopyManager.new(connection)
11 stream = copy_manager.copy_in("COPY mytable(a, b) FROM STDIN WITH CSV")
12
13 begin
14 (1..10000).each do |n|
15 # Don’t forget we’re streaming CSV data, thus each row/line MUST be terminated with a newline
16 row = "#{n},#{2*n}\n".to_java_bytes
17 stream.write_to_copy(row, 0, row.length)
18 end
19 rescue
20 stream.cancel_copy
21 raise
22 else
23 stream.end_copy
24 end
25 end
26 end
27
28 puts time

This will execute a single round trip to the database server: you’ll pay the latence cost only once.

On an unrelated note, this is the first time ever I use an else clause on a begin/rescue. If an exception is raised, we want to cancel the copy (the rescue clause), but on the other hand, if nothing is raised, we want to end the copy (the else clause). One or the other must happen, but not both.

If you’re curious what difference bulk copying makes, here are the benchmark results:

10000 INSERT statements
  7.012000   0.000000   7.012000 (  7.012000)

1 COPY FROM STDIN statement
  0.848000   0.000000   0.848000 (  0.848000)

The numbers speak for themselves: 8× faster. Not too shabby, and remember this ratio will simply increase as the number of rows increases.

See all 2 articles in jruby

Session 2 articles

A bit more than a year ago, I wrote about model serialization in session, in which I advocated to implement dump and load methods in your model to only marshal the object’s ID instead of the full object’s properties.

Today, I found about the object_id_session plugin, which has the same purpose, but a different implementation.

See all 2 articles in session

Liquid 2 articles

Well, it seems I jumped the gun. In Security issue in Liquid::Template, I thought I had a found a problem with the Liquid template engine. Instead, I should have looked more closely at what I do:

app/controllers/pages_controller.rb

1 class PagesController < ApplicationController
2 def show
3 # …
4 render(:inline => @page.render, :layout => false)
5 end
6 end

The details can be found at #render on the Ruby on Rails API. Seems like it’s time for us to switch to using render :text.

I am sorry for any scare I caused. If I had run a separate test case, I’d have immediately seen I was in error, and not Liquid.

See all 2 articles in liquid

Linux 2 articles

I just posted about my problem with the English US keyboard not having a pipe character ? Well, I removed the French Canadian keyboard layout from my configuration, and now I have my pipe character back. That’s not very nice, since I won’t be able to type in French anymore, but at least, I can program again.

Thanks to dvorak keyboard layout but keyboard shortcuts are in qwrty for the idea.

See all 2 articles in linux

Hobo 2 articles

What? You’re still generating migrations manually? I don’t anymore. I have tools which do it for me.

Hobofields is a set of extensions to ActiveRecord that gives us lots of goodies, including the ability to generate migrations automatically. The key to achieving this automatic property declaration is schema declaration in the model.


1 class Post < ActiveRecord::Base
2 fields do
3 title :string
4 published_at :datetime
5 timestamps
6 end
7 end

If you ever used DataMapper, the idea is similar. Where Hobofields shines though is in reading your model and inferring many, many conventions:


1 class Comment < ActiveRecord::Base
2 belongs_to :post
3 end

Running the hobo_migration generator results in:


1 $ script/generate hobo_migration
2
3 -———- Up Migration -———-
4 add_column :comments, :post_id, :integer
5
6 add_index :comments, [:post_id]
7 -———————————————-
8
9 -———- Down Migration -——-
10 remove_column :comments, :post_id
11
12 remove_index :comments, :name => :index_comments_on_post_id rescue ActiveRecord::StatementInvalid
13 -———————————————-

If you use acts_as_list, Hobofields also knows to add a position column (or whatever name is needed):


1 class Comment < ActiveRecord::Base
2 acts_as_list
3 end

Running the migration generator, you get:


1 $ script/generate hobo_migration
2
3 -———- Up Migration -———-
4 add_column :comments, :position, :integer
5 -———————————————-
6
7 -———- Down Migration -——-
8 remove_column :comments, :position
9 -———————————————-
10 What now: [g]enerate migration, generate and [m]igrate now or ©ancel?

Another thing I love about Hobofields is rich type support. Need Textile?


1 class Post < ActiveRecord::Base
2 fields do
3 body :textile
4 end
5 end


1 content = <<EOTEXTILE
2 This is some Textile content
3
4 h1. Automatically formatted
5
6 To your:
7
8 * specifications,
9 * using standard tools

10 EOTEXTILE

11
12 Post.new(:body => content).body.to_html
13 #=> "<p>This is some Textile content</p>\n<h1>Automatically formatted</h1>\n<p>To your:</p>\n<ul><li>specifications,</li>\n<li>using standard tools</li>\n</ul>\n"

Or maybe you need an enumeration?


1 class Comment < ActiveRecord::Base
2 fields do
3 status enum_string(:spam, :ham), :default => :spam, :required => true
4 end
5 end


1 $ script/generate hobo_migration
2
3 -———- Up Migration -———-
4 add_column :comments, :status, :string, :default => "ham", :required => true
5 -———————————————-
6
7 -———- Down Migration -——-
8 remove_column :comments, :status
9 -———————————————-


1 >>Comment.new.status
2 #=> "ham"
3 >> Comment.create!(:status => "foo")
4 #=> ActiveRecord::RecordInvalid: Validation failed: Status must be one of spam, ham

Admittedly, this last example could be better handled with Hobo’s lifecycle (state machine) implementation, but still, it’s nice that the validation already exists. Using :null =&gt; false, :required =&gt; true creates the appropriate validation as well:


1 class Comment < ActiveRecord::Base
2 fields do
3 author :string, :required
4 email :string, :required
5 status enum_string(:spam, :ham), :required => true, :default => "ham"
6 timestamps
7 end
8 end

And Hobofields is smart enough to know when no changes are required:


1 $ script/generate hobo_migration
2 Database and models match — nothing to change

But with this model definition, we now have new validation automatically:


1 >> Comment.create!
2 #=> ActiveRecord::RecordInvalid: Validation failed: Author can’t be blank, Email can’t be blank

Oh wait, just realized that the email address should be an email address:


1 class Comment < ActiveRecord::Base
2 fields do
3 email :email_address, :required
4 end
5 end


1 >> Comment.create!(:email => "a")
2 #=> ActiveRecord::RecordInvalid: Validation failed: Author can’t be blank, Email is invalid

I really enjoy Hobofields for all the goodies it brings to the table. I haven’t even started on all the automatic scopes goodies, the lifecycle methods (which I haven’t used yet), or scoped associations.

Until we meet again tomorrow, I suggest you read Hobofields Reference, as well as Hobofields rich types to get the full goods.

See all 2 articles in hobo

Serialization 2 articles

A bit more than a year ago, I wrote about model serialization in session, in which I advocated to implement dump and load methods in your model to only marshal the object’s ID instead of the full object’s properties.

Today, I found about the object_id_session plugin, which has the same purpose, but a different implementation.

See all 2 articles in serialization

Budgetapp 2 articles

I have worked some more on my home budget planner application. The current version is implemented on Seaside, and it’s doing great. I haven’t had as much time as I wanted, but it’s coming along nicely. I have some formal training in accounting practices, and I treat my house as being a business: we have income and expense accounts, equity for ourselves, and assets (our bank account) as well as liability accounts (credit cards and bank loans).



BudgetApp’s administration tab. Click for larger view.

This is the administration tab. This is where you add and change accounts. Not much more to show here.



BudgetApp’s budget tab. Click for larger view.

The budget tab. This is where you actually set your budget targets for the month. Historical data is kept around, and you can immediately see when your budget is under the actual value.



BudgetApp’s real tab. Click for larger view.

This is the least polished of the tabs yet, and ironically, this is where most of the work is going to be done. I’ll need to use the application for a bit before I can determine the exact interface I want. I’m thinking of having a couple of panels that will allow the user to say what kind of transaction occurred: paid, bought, reimbursed, transferred, etc.

That’s the state of affairs at revision 9 on the Monticello repository.

Differences with Rails

I haven’t actually started doing any work on the Rails side of things, but there is one thing I did notice: I find it easier to segregate my work in change sets in the file world versus when working in the image-world. For example, revision 8 includes changes to a couple of classes, and none are related: stylistic changes in the budget and admin tabs, plus my initial stab at the real tab. Had I been using Rails, I would have committed a couple of files here and there multiple times, and that would be it.

I am aware of Monticello’s “add to current change set” and “remove from current change set”, but have not dared using them yet. I’m not exactly sure what these options will do, and most importantly, I am afraid of losing work. That probably won’t happen, but there’s this nagging feeling deep down…

Anyway, next step is to generate real transactions from the real tab. More on this next week !

See all 2 articles in budgetapp

Trick 2 articles

Well, I just got the answer I needed: how to clone just the tip of a remote Git repository ? Here’s how:

Sometimes you just want to distribute the source code without its history, and that’s where git-archive comes in.

git archive &lt;tree-ish&gt; &gt; my_new_archive.tar

Kate Rhodes in Getting just the tip of a Git repo

I think I might use this in Piston. I’m not too sure yet, as I haven’t implemented piston-git, but it looks like a promising candidate.

See all 2 articles in trick

Mor9 2 articles

Yesterday (Tuesday May 20th, 2008), I presented at Montreal on Rails. I made a short and sweet presentation on Mephisto, and how I refactored it to support both Akismet and Defensio.

You can grab the slides for “Refactoring to Patterns: How Mephisto went from a single engine Lada to a multi-engine jet fighter”/2008/05/21/refactoring-to-patterns.pdf (PDF).

References

Design Patterns

Other interesting patterns that I used in Mephisto, which I briefly talked about, but haven’t mentioned in the slides at all:

Refactoring

  • Refactoring to Patterns

Other things I talked about

See all 2 articles in mor9

Tips’n’tricks 2 articles

If like me you didn’t know, you can do that with Capistrano:


1 $ cap HOSTS=web0.myapp.com deploy:update_code
2 cap ROLES=web,app deploy

Not terribly useful for the tasks demonstrated above, but what about this ?


1 $ cap ROLES=app monit:summary

Incidentally, where are these documented ? I dimly remembered reading this somewhere once, and tried it. Works like a charm!

See all 2 articles in tips’n’tricks

Markup 2 articles

Well, it seems I jumped the gun. In Security issue in Liquid::Template, I thought I had a found a problem with the Liquid template engine. Instead, I should have looked more closely at what I do:

app/controllers/pages_controller.rb

1 class PagesController < ApplicationController
2 def show
3 # …
4 render(:inline => @page.render, :layout => false)
5 end
6 end

The details can be found at #render on the Ruby on Rails API. Seems like it’s time for us to switch to using render :text.

I am sorry for any scare I caused. If I had run a separate test case, I’d have immediately seen I was in error, and not Liquid.

See all 2 articles in markup

Bundler 2 articles

I just pushed a change for my teammates: we’re now using Bundler on our Rails 2.3.5 application. Things went rather well, but our continuous testing server choked a bit on that.

First, I had forgotten to run bundle install after pulling changes. Then I had to ignore our cucumber environment, because it depends on rb-appscript which is a native gem on Mac OS X. Our CT machine is a Linux box, hence the change. Then, bundle install would fail installation because it didn’t have sudo access:


1 Fetching git://github.com/bloom/rails.git
2 Fetching git://github.com/francois/active_url
3 Fetching git://github.com/francois/safariwatir.git
4 Fetching source index for http://rubygems.org/
5 sudo: no tty present and no askpass program specified
6 sudo: no tty present and no askpass program specified
7 Using rake (0.8.7)
8 sudo: no tty present and no askpass program specified
9 sudo: no tty present and no askpass program specified
10 /usr/lib64/ruby/site_ruby/1.8/rubygems/format.rb:38:in `from_file_by_path’: Cannot load gem at [/usr/lib64/ruby/gems/1.8/cache/SystemTimer-1.2.gem] in /home/francois/adgear-admin (Gem::Exception)
11 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/source.rb:77:in
`
fetch
12 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/installer.rb:45:in `run

13 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/spec_set.rb:12:in `each’
14 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/spec_set.rb:12:in
`
each
15 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/installer.rb:44:in `run

16 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/installer.rb:8:in `install’
17 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/cli.rb:185:in
`
install
18 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/vendor/thor/task.rb:22:in `send

19 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/vendor/thor/task.rb:22:in `run’
20 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/vendor/thor/invocation.rb:118:in
`
invoke_task
21 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/vendor/thor.rb:246:in `dispatch

22 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/lib/bundler/vendor/thor/base.rb:389:in `start’
23 from /usr/lib64/ruby/gems/1.8/gems/bundler-1.0.0.rc.5/bin/bundle:13
24 from /usr/bin/bundle:19:in
`
load
25 from /usr/bin/bundle:19
26

I was busy writing a message to the Ruby Bundler Google Group when I remembered bundle install has an optional parameter which specifies where the bundle should be installed. I changed my CT script to run bundle install --without=cucumber vendor/gems and bundle did it’s magic, and my CT box started running tests.

Much rejoicing ensued.

For the record, here’s our testing script:


1 #!/bin/bash
2 cd /home/francois/adgear-admin
3
4 if [ z $1 ] ; then
5 echo "Can’t proceed if I don’t know which branch to test"
6 exit 1
7 fi
8
9 TSTAMP=`date +%Y
%m-%d-%H:%M`
10 BRANCH=$1
11 LOG=/home/francois/build-logs/$BRANCH-$TSTAMP.log
12
13 (
14 git fetch origin && \
15 git reset -hard origin/$BRANCH && \
16 echo ======= && \
17 git show && \
18 ( bundle install —without=cucumber vendor/cache || true ) && \
19 nice rake db:adgear:production:update parallel:prepare parallel:test
20 ) > $LOG 2>&1
21
22 if [ "$?" = "0" ] ; then
23 /usr/bin/curl -s -o /dev/null -d "{\"message\":{\"body\":\"Test Suite Passed -
$BRANCH — http://ct.local:9001/logs/$BRANCH/$TSTAMP\"}}" HContent-Type:application/json -u 12345:X http://oursite.campfirenow.com/room/12345/speak.json
24 else
25 /usr/bin/curl -s -o /dev/null -d "{\"message\":{\"body\":\"Test Suite FAILED -
$BRANCH — http://ct.local:9001/logs/$BRANCH/$TSTAMP\"}}" -HContent-Type:application/json -u 12345:X http://oursite.campfirenow.com/room/12345/speak.json
26 fi
27
28 # vi: sw=8 ts=8 expandtab

This script is run from cron.

See all 2 articles in bundler

Amazon 2 articles

Starting from a fresh Rails application (I’m using 2.0.2), install AttachmentFu:


1 script/plugin install http://svn.techno-weenie.net/projects/plugins/attachment_fu/

Edit config/amazon_s3.yml and put this:

config/amazon_s3.yml

1 development:
2 bucket_name: amazon-sqs-development-yourname
3 access_key_id: "your key"
4 secret_access_key: "your secret access key"
5 queue_name: amazon-sqs-development-resizer-yourname

queue_name is new. AttachmentFu does not require this, but we are going to reuse the file from our own code, so better put all configuration in the same place.

Generate a scaffolded Photo model using:


1 $ script/generate scaffold photo filename:string size:integer content_type:string width:integer height:integer parent_id:integer thumbnail:string

Edit app/views/photos/new.erb.html and replace everything with this:

app/views/photos/new.erb.html

1 <h1>New photo</h1>
2
3 <%= error_messages_for :photo >
4
5 < form_for(@photo, :html => {:multipart => true}) do |f| >
6 <p>
7 <label for="photo_uploaded_data">File:</label>
8 <= f.file_field :uploaded_data >
9 </p>
10
11 <p>
12 <= f.submit "Create" >
13 </p>
14 < end >
15
16 <= link_to ‘Back’, photos_path %>


What we did here is simply tell Rails to use a multipart encoded form, and to only provide us with a single file upload field.

Edit app/models/photo.rb and add the AttachmentFu plugin configuration:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 has_attachment :content_type => :image, :storage => :s3
3 validates_as_attachment
4 end

Start your server and confirm you can upload a file. No thumbnails were generated as we did not configure any thumbnailing to do. We don’t actually want AttachmentFu to handle that, so we can’t just specify it in the has_attachment call.

To use RightScale’s AWS SQS component, we have to configure it with the access key and secret access key. Add this to the end of the Photo class:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def queue
3 self.class.queue
4 end
5
6 class << self
7 def queue
8 # This creates the queue if it doesn’t exist
9 queue</span> ||= sqs.queue(aws_config[<span class="s"><span class="dl">&quot;</span><span class="k">queue_name</span><span class="dl">&quot;</span></span>]) <span class="no"><strong>10</strong></span> <span class="r">end</span> <span class="no">11</span> <span class="no">12</span> <span class="r">def</span> <span class="fu">sqs</span> <span class="no">13</span> <span class="iv">sqs ||= RightAws::Sqs.new(
14 aws_config["access_key_id"], aws_config["secret_access_key"],
15 :logger => logger)
16 end
17
18 def aws_config
19 return aws_config</span> <span class="r">if</span> <span class="iv">aws_config
20
21 aws_config</span> = <span class="co">YAML</span>.load(<span class="co">File</span>.read(<span class="co">File</span>.join(<span class="co">RAILS_ROOT</span>, <span class="s"><span class="dl">&quot;</span><span class="k">config</span><span class="dl">&quot;</span></span>, <span class="s"><span class="dl">&quot;</span><span class="k">amazon_s3.yml</span><span class="dl">&quot;</span></span>))) <span class="no">22</span> <span class="iv">aws_config = aws_config</span>[<span class="co">RAILS_ENV</span>] <span class="no">23</span> raise <span class="co">ArgumentError</span>, <span class="s"><span class="dl">&quot;</span><span class="k">Missing </span><span class="il"><span class="idl">#{</span><span class="co">RAILS_ENV</span><span class="idl">}</span></span><span class="k"> configuration from config/amazon_s3.yml file.</span><span class="dl">&quot;</span></span> <span class="r">if</span> <span class="iv">aws_config.nil?
24 @aws_config
25 end
26 end
27 end

#aws_config is a method that reads the configuration. #sqs is a method that provides access to an instance of RightScale::Sqs, pre-configured with the correct access keys. #queue uses #sqs to get or create a named queue. There’s also an instance version of #queue, to ease our code later on.

Let’s add the request sending:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def send_resize_request
3 # Don’t send a resize request for thumbnails
4 return true unless self.parent_id.blank?
5
6 params = Hash.new
7 params[:id] = self.id
8 params[:sizes] = Hash.new
9 params[:sizes][:square] = "75×75"
10 params[:sizes][:thumbnail] = "100x"
11
12 begin
13 queue.push(params.to_yaml)
14 rescue
15 logger.warn {"Unable to send resize request. Error: #{$!.message}"}
16 logger.warn {$!.backtrace.join("\n")}
17
18 # Don’t raise the error so the request goes through.
19 # We don’t want the user to see a 500 error because
20 # we can’t talk to Amazon.
21 end
22 end
23 end

Now, this is getting interesting. AttachmentFu knows if the current model is a thumbnail or not by looking at parent_id. If it’s nil, we are the parent, else we are a thumbnail. We do the same thing here.

Then, we setup a couple of parameters to send to the resizer. Notice we send the actual thumbnail sizes in the message itself.

Next, we do the most important part: queue.push. This sends a message string (limited to 256 KiB) to Amazon SQS, and returns. If there is an error, we don’t actually want to prevent the request from completing, so we rescue any exceptions and log them. If you have the ExceptionNotifier plugin installed, this is a good place to log to it.

Now that we have a way to send the resize request, we have to execute it at some point. The controller is not the right place to do it. If you create Photo models from more than one controller, you’re bound to forget to call #send_resize_request. It’s better to do it in an #after_create callback, which we’ll do with a single line:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 after_create :send_resize_request
3 end

Next, we have to receive the messages. So, we write a new method in Photo:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 class << self
3 def fetch_and_thumbnail
4 messages = queue.receive_messages(20)
5 return if messages.blank?
6
7 logger.debug {"==> Photo\#fetch_and_thumbnail — received #{messages.size} messages"}
8 messages.each do |message|
9 params = YAML.load(message.body)
10 photo = Photo.find_by_id(params[:id])
11 if photo.blank? then
12 # The Photo was deleted before we got a chance to thumbnail it.
13 # We must delete the message, or we’ll always get it afterwards.
14 message.delete
15 next
16 end
17
18 photo.generate_thumbnails(params[:sizes])
19 message.delete
20 end
21 end
22 end
23 end

The first thing we do is see if there are any messages. The call to #queue is the helper method we defined earlier on. We ask to receive up to 20 messages at a time. If there were no messages, we simply return.

Then, for each message, we have to process it, so we iterate over each message, retrieving the original parameters Hash. The important thing to do is to delete the message after we have processed it, or else the message will still be visible next time around.

#generate_thumbnails is important, but uninteresting in this discussion.

See all 2 articles in amazon

Version 2 articles

Daniel Berlinger asks a good question about my Sharing models between two Rails applications using Piston asking:

Without having a moment to think about it… there must be other ways to accomplish this (in Rails) without Piston, and other Subversion trickery. No?

Daniel Berlinger

I know of only one way to do it: create a plugin to hold the models, and load this plugin into all of the applications that need the shared models.

See all 2 articles in version

Recovery 1 articles

Ouch! That hurts! Thurdsay (yesterday) I finished setting up new DB instances for XLsuite. All was well, and the servers performed admirably well for 24 hours. Then, suddenly, the LVM partition that held /var/lib/mysql went away…

Around 18:08 UTC (11:08:09 PDT), our main MySQL database server went down. Luckily, yesterday (Thurdsay), I had just replaced our whole DB infrastructure to have a replicated master/slave setup. It took us 15 minutes to notice that the sites were down, and another 20 minutes to execute a database failover. By 18:50 UTC (11:50 PDT), things were back to normal.

François Beausoleil

Unfortunately, I did not have the scripts in place yet to do the database failover, so it was a manual process. It wasn’t too hard, but I did have to remember one thing:


1 Mysql::Error: The MySQL server is running with the —read-only option so it cannot execute this statement: INSERT INTO sessions (`updated_at`, `sessid`, `data`) VALUES

In High Performance MySQL, the authors recommend, in Chapter 8:

On the slave, we recommend enabling the following configuration options:

skip_slave_start
read_only

Double oops! Anyway, the advice in the book was invaluable to me. If you manage MySQL and don’t already have the book, I highly suggest you buy it.

See all 1 articles in recovery

Deprecation 1 articles

Rails deprecations are a nice thing. They allow us to know what transition path to use, instead of being left in the dark. With deprecated functionality being removed from Rails, it’s important to know these things.

If you have a decent test coverage, you might want to add this to your config/environments/test.rb:

config/environments/test.rb

1 class DeprecatedFunctionality < SyntaxError; end
2 ActiveSupport::Deprecation.behavior = Proc.new {|message, callstack|
3 raise DeprecatedFunctionality, message
4 }

This will raise an exception instead of simply reporting the warning to the console. If you have CruiseControl.rb installed, the messages will show up as errors there too. This is a quick and easy way to force yourself to actually use the new functionality.

See all 1 articles in deprecation

Events 1 articles

On August 7th, I’ll be at Montreal on Rails. Will you ?

See all 1 articles in events

Quirks 1 articles


1 $ git status
2 # On branch master
3 nothing to commit (working directory clean)
4 $ echo $?
5 1

Shouldn’t a successful git status in a git repository return a status code of 0 ? Doing it in a random folder returns a sensible value:


1 $ git status
2 fatal: Not a git repository
3 Failed to find a valid git directory.
4 $ echo $?
5 128

Can anybody shed some light on this ?

I found Possible bug in ‘git status’ exit code is 1 instead of 0. What is the rational for this ? It goes against everything Unix ?!?

See all 1 articles in quirks

Restclient 1 articles

I just merged Rick Olson’s (technoweenie) rest-client multipart_streaming branch with RestClient 1.0.4. You can see the results at francois/rest-client. I sent a pull request to Adam, so if he pulls it in, everyone will benefit from this code.

See all 1 articles in restclient

Conversion 1 articles

I just found about Media Convert, a free online tool that converts files from one format to another. I converted an FLV to SWF, and it worked flawlessly. Their converters also convert from many other formats. Hope this helps someone.

See all 1 articles in conversion

Active-record 1 articles

At Bloom, we have a largish Rails application: 12 000 lines of model code, alone. Yesterday, we found a major problem, and the cause was very non-obvious.

Bloom Digital Platforms has a product called AdGear. AdGear is an ad distribution platform. The obvious models are Ad and Campaign: a Campaign has many ads. From a UI perspective, it’s nice to show how many active ads are in the campaign. In the campaign’s index view, we have something like this:


1 <%# app/views/campaigns/campaign.html.erb >
2 <tr>
3 <td><
= link_to(h(campaign.name), campaign) ></td>
4 <td><
= pluralize(campaign.active_ad_units
count, "active ads") %></td>
5 </tr>
6

Notice how we’re using an attribute named active_ad_units_count. This is a cached value. We keep this value updated by doing the following:


1 class Ad < ActiveRecord::Base
2 belongs_to :campaign
3 after_save :update_campaign_cached_values
4
5 private
6
7 def update_campaign_cached_values
8 campaign.update_cached_values
9 end
10 end
11
12 class Campaign < ActiveRecord::Base
13 def update_cached_values
14 update_attribute(:active_ad_units_count, ad_units.active.count)
15 end
16 end

Pretty reasonable, you’ll say. And I agree: good names explain the intent, without needing to read all the details.

Let’s throw a monkey wrench in there:


1 class Campaign < ActiveRecord::Base
2 before_save :ensure_consistency
3
4 attr_accessor :conditions
5
6 def conditions
7 return conditions</span> <span class="r">if</span> <span class="iv">conditions
8 @conditions =|| serialized_conditions ? YAML.load(serialized_conditions) : nil
9 end
10
11 private
12
13 # Method renamed to protect the innocent
14 def ensure_consistency
15 if some_condition then
16 self.serialized_conditions = conditions ? conditions.to_yaml : nil
17 return true
18 end
19
20 false
21 end
22 end

Can you spot the issue? Note this code is in Campaign, not Ad. It’s pretty obvious when boiled down to 20 lines of code. When you’re wading through 12 000, it’s pretty tough.

If you read the documentation for #before_save, you’ll notice this gem:


1 # == Canceling callbacks
2 #
3 # If a <tt>before_*</tt> callback returns false, all the later callbacks and the associated action are cancelled. If an <tt>after_*</tt> callback returns
4 # false, all the later callbacks are cancelled. Callbacks are generally run in the order they are defined, with the exception of callbacks
5 # defined as methods on the model, which are called last.

Excerpt from activerecord/lib/active_record/callbacks.rb

In the example above, some_condition might return false, which would cancel the save, but let’s unwind the stack:

  • AdsController#update
  • Ad#update_campaign_cached_values
  • Campaign#update_cached_values
  • ActiveRecord::Base#update_attribute
  • Campaign#ensure_consistency – returns false

Because we were calling #update_attribute, which returns false when it fails to update the object, we ended up not updating the Campaign’s cached values. And we ended up with a stale cache. And ads which didn’t get updated. A careful tower of convoluted code tumbled down upon us.

Why did this happen? It’s obvious the developer who wrote the #ensure_consistency method didn’t know about ActiveRecord’s #serialize, which did exactly what he wanted to. He also didn’t know about the dangers of returning false from a callback. These two point to relative inexperience / lack of knowledge about Rails & ActiveRecord.

We’re obviously not the first ones to hit this kind of problem:

James Golick also ranted off about something similar a while back: Crazy, Heretical, and Awesome: The Way I Write Rails Apps

This points to a misfeature of ActiveRecord: a seemingly innocuous change had unintended consequences in a very unrelated portion of the code. Rails has a lot of magic / implicits. You have to be very careful. Ignorance is no excuse. The more I think about it, the more I like the service / command oriented way James offers, and it boils down to the discussion we had at the last Montreal.rb: implicit (Rails/Ruby) vs explicit (Django/Python).

See all 1 articles in active-record

Language-design 1 articles

Image a Ruby where if/then/else isn’t available:


1 class Account
2 # This method returns the balance in words, negative or positive, ready for display in the UI
3 def balance_in_words
4 # Err… How do I do that?
5 end
6 end

If you were a strict Object-Oriented person, you’d do it this way:


1 class PositiveBalanceAccount
2 def balance_in_words
3 "Positive"
4 end
5 end
6
7 class NegativeBalanceAccount
8 def balance_in_words
9 "Negative"
10 end
11 end

But then, your objects would have to change class whily-nilly. Pretty darn hard. But multi-method dispatching gives us a nice solution:


1 class Account
2 def initialize(balance)
3 balance</span> = balance <span class="no"> 4</span> <span class="r">end</span> <span class="no"> <strong>5</strong></span> <span class="no"> 6</span> defmulti <span class="sy">:balance_in_words</span>, <span class="no"> 7</span> lambda { <span class="iv">balance < 0 } => "Negative",
8 lambda { @balance > 0 } => "Positive"
9 end
10
11 Account.new(15).balance_in_words # => "Positive"
12 Account.new(-5).balance_in_words # => "Negative"

For the observant amongst you, you might have noticed I forgot the nil balance case. This is really a programming error, so it should be treated as such:


1 Account.new(0).balance_in_words
2 lib/defmulti.rb:46:in `balance_in_words’: #<Account:0×197222c @balance=0> received balance_in_words but did not have a guard clause that matched and no else clause. (Defmulti::MissingGuardClause)
3 from test.rb:16
4

For the even more observant, yes, there exists an implementation that does exactly what I have described above. It’s called defmulti, it’s a gem, and it’s on GitHub: http://github.com/francois/defmulti.

What is this useful for?

This library is a thought experiment. When you lose the familiar tools, what can you do? Of course, this library works atop the existing Ruby implementation and to be truly useful, syntax would have to be provided to make this much less verbose. It’s interesting nonetheless to see what can be done without the syntax extensions.

Anyway, what would you use this for? Multi-method dispatching is a tool that helps writing code without conditionals. The conditionals are specified outside the block of code that executes. The examples above are pretty thin, but looking at Java.next #3: Dispatch, I can provide another solution to his Ruby example:


1 defmulti :letter_grade,
2 lambda {|grade| (90..100).include?(grade) || grade == "A"} => "A",
3 lambda {|grade| (8090).include?(grade) || grade == "B"} => "B",
4 lambda {|grade| (7080).include?(grade) || grade == "C"} => "C",
5 lambda {|grade| (6070).include?(grade) || grade == "D"} => "D",
6 lambda {|grade| ( 060).include?(grade) || grade == "F"} => "F"
7
8 letter_grade 60 # => "D"
9 letter_grade "A" # => "A"
10 letter_grade nil # => Defmulti::MissingGuardClause

Again, I’m struck by the clunky syntax, but if we ignore that for a second, could this be even better than Stuart Halloway’s example? Ruby’s case statement is very, very powerful and very easy to use. Is this useful? Not at the moment. But it’s a thought experiment I thought I’d throw out there.

See all 1 articles in language-design

Woes 1 articles

Okay, I have a use case which I think shouldn’t be too hard. Jean-François, my Montreal on Rails buddy, gave me a good hand, but I think there’s some interaction going on between Github and git-svn that causes friction…

I have a new project which I want to track using Git, but I also want to keep a push-only Subversion repository on Rubyforge.

So, I started like any old project:


1 $ svn checkout svn+ssh://fbos@rubyforge.org/var/svn/theproject
2 $ svn mkdir tags branches
3 $ newgem theproject
4 $ mv theproject trunk
5 $ svn add trunk
6 $ svn commit -m "New project layout"

Then, I imported that into a fresh Git repository:


1 $ git svn clone svn+ssh://fbos@rubyforge.org/var/svn/theproject -T trunk -t tags -b branches theproject

I then started coding and commiting using Git. All was well, and I was happy. Then, I pushed to Subversion, like any good developer should:


1 $ git svn dcommit

That worked well enough. Then, I was ready to have a public Git repository. I went to Github, where I have an account, and created a new repository. I then did:


1 $ git remote add github git@github.com:francois/theproject.git
2 $ git push github master

That gave me a couple of errors:


1 error: remote ‘refs/heads/master’ is not a strict subset of local ref
2 ‘refs/heads/master’. maybe you are not up-to-date and need to pull first?
3 error: failed to push to ‘git@github.com:francois/theproject.git’

After some discussion with Jean-François, he told me to have a separate branch on which to do development, and merge to master when I am ready to dcommit. So, I did:


1 $ git checkout -b mainline
2 # code again
3 $ git checkout master
4 $ git merge mainline
5 $ git svn dcommit

That works well enough. Now I want to push my mainline to Github. So I do:


1 $ git remote add github git@github.com:francois/theproject.git
2 $ git push github mainline
3 $ git push github mainline
4 updating ‘refs/heads/mainline’
5 from 0000000000000000000000000000000000000000
6 to 31d6eee71b00f829b8568937ab3adaaa8831205c
7 Generating pack…
8 Done counting 201 objects.
9 Deltifying 201 objects…
10 100% (201/201) done
11 Writing 201 objects…
12 100% (201/201) done
13 Total 201 (delta 100), reused 0 (delta 0)
14 refs/heads/mainline: 0000000000000000000000000000000000000000 -> 31d6eee71b00f829b8568937ab3adaaa8831205c

Great! It works. Except… The repository page on Github still shows I need to create the repository. Just for fun, without doing anything else I created a new empty Git repository, touched README, added that, remoted Github and pushed. Voilà, Github showed me the repository, with README being added.

Can anyone shed some light on this ? Here’s a reproduction recipe:


1 $ cat repro.sh
2 #!/bin/sh
3
4 rm -rf repos wc theproject
5 svnadmin create repos
6 svn checkout file:///`pwd`/repos wc
7 cd wc
8 svn mkdir tags branches
9 newgem theproject
10 mv theproject trunk
11 svn add trunk
12 svn commit -m "Initial revision"
13 cd ..
14 git svn clone file:///`pwd`/repos theproject
15 cd theproject
16 echo "The new revision" > README
17 git add README
18 git commit -a -m "New README content"
19 git checkout -b mainline
20 git checkout master
21 git svn dcommit
22 git checkout mainline
23 echo "More content" >> README
24 git commit -a -m "More README goodness"
25 git remote add github git@github.com:francois/theproject.git
26 git push github mainline
27 git checkout master
28 git merge mainline
29 git svn dcommit

And here’s a sample bash +x run:


1 + rm rf repos wc theproject
2 + svnadmin create repos
3 + pwd
4 + svn checkout file:////home/francois/src/repro/repos wc
5 Checked out revision 0.
6 + cd wc
7 + svn mkdir tags branches
8 A tags
9 A branches
10 + newgem theproject
11 create
12 create config
13 create doc
14 create lib
15 create log
16 create script
17 create tasks
18 create test
19 create tmp
20 create lib/theproject
21 create History.txt
22 create License.txt
23 create Rakefile
24 create README.txt
25 create setup.rb
26 create lib/theproject.rb
27 create lib/theproject/version.rb
28 create config/hoe.rb
29 create config/requirements.rb
30 create log/debug.log
31 create tasks/deployment.rake
32 create tasks/environment.rake
33 create tasks/website.rake
34 create test/test_helper.rb
35 create test/test_theproject.rb
36 dependency install_website
37 create website/javascripts
38 create website/stylesheets
39 exists script
40 exists tasks
41 create website/index.txt
42 create website/index.html
43 create script/txt2html
44 force tasks/website.rake
45 dependency plain_theme
46 exists website/javascripts
47 exists website/stylesheets
48 create website/template.rhtml
49 create website/stylesheets/screen.css
50 create website/javascripts/rounded_corners_lite.inc.js
51 dependency install_rubigen_scripts
52 exists script
53 create script/generate
54 create script/destroy
55 create Manifest.txt
56 readme readme
57 Important
58 =====
59
60 * Open config/hoe.rb
61 * Update missing details (gem description, dependent gems, etc.)
62 + mv theproject trunk
63 + svn add trunk
64 A trunk
65 A trunk/Manifest.txt
66 A trunk/History.txt
67 A trunk/doc
68 A trunk/setup.rb
69 A trunk/tmp
70 A trunk/test
71 A trunk/test/test_theproject.rb
72 A trunk/test/test_helper.rb
73 A trunk/tasks
74 A trunk/tasks/deployment.rake
75 A trunk/tasks/website.rake
76 A trunk/tasks/environment.rake
77 A trunk/lib
78 A trunk/lib/theproject.rb
79 A trunk/lib/theproject
80 A trunk/lib/theproject/version.rb
81 A trunk/script
82 A trunk/script/txt2html
83 A trunk/script/destroy
84 A trunk/script/generate
85 A trunk/Rakefile
86 A trunk/website
87 A trunk/website/stylesheets
88 A trunk/website/stylesheets/screen.css
89 A trunk/website/javascripts
90 A trunk/website/javascripts/rounded_corners_lite.inc.js
91 A trunk/website/index.txt
92 A trunk/website/template.rhtml
93 A trunk/website/index.html
94 A trunk/log
95 A trunk/log/debug.log
96 A trunk/config
97 A trunk/config/requirements.rb
98 A trunk/config/hoe.rb
99 A trunk/License.txt
100 A trunk/README.txt
101 + svn commit -m ‘Initial revision’
102 Adding branches
103 Adding tags
104 Adding trunk
105 Adding trunk/History.txt
106 Adding trunk/License.txt
107 Adding trunk/Manifest.txt
108 Adding trunk/README.txt
109 Adding trunk/Rakefile
110 Adding trunk/config
111 Adding trunk/config/hoe.rb
112 Adding trunk/config/requirements.rb
113 Adding trunk/doc
114 Adding trunk/lib
115 Adding trunk/lib/theproject
116 Adding trunk/lib/theproject/version.rb
117 Adding trunk/lib/theproject.rb
118 Adding trunk/log
119 Adding trunk/log/debug.log
120 Adding trunk/script
121 Adding trunk/script/destroy
122 Adding trunk/script/generate
123 Adding trunk/script/txt2html
124 Adding trunk/setup.rb
125 Adding trunk/tasks
126 Adding trunk/tasks/deployment.rake
127 Adding trunk/tasks/environment.rake
128 Adding trunk/tasks/website.rake
129 Adding trunk/test
130 Adding trunk/test/test_helper.rb
131 Adding trunk/test/test_theproject.rb
132 Adding trunk/tmp
133 Adding trunk/website
134 Adding trunk/website/index.html
135 Adding trunk/website/index.txt
136 Adding trunk/website/javascripts
137 Adding trunk/website/javascripts/rounded_corners_lite.inc.js
138 Adding trunk/website/stylesheets
139 Adding trunk/website/stylesheets/screen.css
140 Adding trunk/website/template.rhtml
141 Transmitting file data ……………………
142 Committed revision 1.
143 + cd ..
144 +
pwd
145 + git svn clone file:////home/francois/src/repro/repos theproject
146 Initialized empty Git repository in .git/
147 A trunk/History.txt
148 A trunk/test/test_helper.rb
149 A trunk/test/test_theproject.rb
150 A trunk/License.txt
151 A trunk/log/debug.log
152 A trunk/Rakefile
153 A trunk/setup.rb
154 A trunk/website/template.rhtml
155 A trunk/website/index.txt
156 A trunk/website/javascripts/rounded_corners_lite.inc.js
157 A trunk/website/index.html
158 A trunk/website/stylesheets/screen.css
159 A trunk/Manifest.txt
160 A trunk/script/txt2html
161 A trunk/script/destroy
162 A trunk/script/generate
163 A trunk/config/requirements.rb
164 A trunk/config/hoe.rb
165 A trunk/tasks/deployment.rake
166 A trunk/tasks/website.rake
167 A trunk/tasks/environment.rake
168 A trunk/lib/theproject/version.rb
169 A trunk/lib/theproject.rb
170 A trunk/README.txt
171 W: empty_dir: branches
172 W: +empty_dir: tags
173 W: +empty_dir: trunk/doc
174 W: +empty_dir: trunk/tmp
175 r1 = 01d09dc543711efb5bbd39e71ea2d1fe12516926 (git-svn)
176
177 Checked out HEAD:
178 file:////home/francois/src/repro/repos r1
179 + cd theproject
180 + echo ‘The new revision’
181 + git add README
182 + git commit -a -m ‘New README content’
183 Created commit 16eaad5: New README content
184 1 files changed, 1 insertions(
), 0 deletions(
)
185 create mode 100644 README
186 + git checkout b mainline
187 Branch mainline set up to track local branch refs/heads/master.
188 Switched to a new branch "mainline"
189 + git checkout master
190 Switched to branch "master"
191 + git svn dcommit
192 A README
193 Committed r2
194 A README
195 r2 = 5b2c01dde36111a37f942d9fb6670689089ad9ed (git-svn)
196 No changes between current HEAD and refs/remotes/git-svn
197 Resetting to the latest refs/remotes/git-svn
198 + git checkout mainline
199 Switched to branch "mainline"
200 + echo ‘More content’
201 + git commit -a -m ‘More README goodness’
202 Created commit 49df2b5: More README goodness
203 1 files changed, 1 insertions(+), 0 deletions(
)
204 + git remote add github git@github.com:francois/theproject.git
205 + git push github mainline
206 error: remote ‘refs/heads/mainline’ is not a strict subset of local ref ‘refs/heads/mainline’. maybe you are not up-to-date and need to pull first?
207 error: failed to push to ‘git@github.com:francois/theproject.git’
208 + git checkout master
209 Switched to branch "master"
210 + git merge mainline
211 Auto-merged README
212 CONFLICT (add/add): Merge conflict in README
213 Automatic merge failed; fix conflicts and then commit the result.
214 + git svn dcommit
215 No changes between current HEAD and refs/remotes/git-svn
216 Resetting to the latest refs/remotes/git-svn
217 README: needs update

See all 1 articles in woes

Employment 1 articles

After a few years as a consultant, I received an offer from Bosko Milekic from Bloom Digital Platforms Inc. Starting on October 5th, I will be a permanent Bloom employee. Marc-André and Gary are moving outside the enterprise, while I’m moving inside.

At Bloom, I will be responsible for the AdGear API, enabling applications to easily connect to their platform for ad serving and management. We even have an open source Ruby client available on GitHub as ad_gear_client.

This will be a welcome change of pace for me.

See you at the next Montreal.rb, at my new office!

See all 1 articles in employment

Repository 1 articles

Well, after reading many interesting articles on Git, I moved Piston’s repository to Gitorious. Clone, fix and extend at will. I will maintain the existing Subversion repository and do releases when fixes come in. I will not maintain Piston itself in any significant ways though.

So, for those of you that expressed an interest in maintaining Piston, go ahead ! Get Git, get coding and publish your repository so I can pull from it.

Piston on Gitorious is available at http://gitorious.org/projects/piston. The mainline (or trunk in Subversion terms) is available as http://gitorious.org/projects/piston/repos/mainline

See all 1 articles in repository

Editor 1 articles

I started evaluating the e Text Editor this morning. At first, I thought I would be in honeymoon, but I quickly realized that was not to be.

See all 1 articles in editor

Humor 1 articles

Humorous Moment

2006-10-20

I use the Money gem quite a bit in my Rails applications. One thing I needed was to often add or substract one penny from an amount. So, I did like all good programmers should do:

lib/money_extensions.rb

1 class Money
2 Penny = Money.new(1).freeze
3 end

Turns out that when using the constant, it ends up looking like this:


1 amount -= Money::Penny

I never knew I’d have a Bond Girl in my Rails code :)

See all 1 articles in humor

Extensions 1 articles

A minor plugin for all of you. This plugin merely adds two new methods to Time and Date:

test/time_extensions_test.rb

1 class TimeExtensionsTest < Test::Unit::TestCase
2 def test_really_in_future
3 assert 1.second.from_now.in_future?
4 end
5
6 def test_in_future_but_past
7 assert !1.second.ago.in_future?
8 end
9
10 def test_really_in_past
11 assert 1.second.ago.in_past?
12 end
13
14 def test_in_past_but_future
15 assert !1.second.from_now.in_future?
16 end
17 end

This makes for much more readable code:


1 if @post.published_at.in_future? then
2 # do something
3 else
4 # do something else
5 end

Installation:


1 $ script/plugin install \
2 svn://svn.teksol.info/svn/rails/plugins/time_extensions \
3 vendor/plugins/time_extensions

See all 1 articles in extensions

Railsrumble-2008 1 articles

Well, the competition is started, but I can’t access my account. I use Technorati as my OpenID provider, but Technorati won’t let me grant RailsRumble permission to read my profile.

Error: You do not have permission to authorize this url. Please sign in to the appropriate account.

Yee ha! The wonders of OpenID! But the saddest fact is that I did authorized RailsRumble at Technorati at least once, since I could sign in and register.

Who said OpenID solved all problems? If I had an account at RailsRumble, this wouldn’t be a problem…

Anyway, that won’t prevent me from kickin’ asses!

See all 1 articles in railsrumble-2008

Conference 1 articles

Because, I am ! RubyFringe is a conference held by Hampton Catlin, the creator of Haml. If you’re going, leave me a note and we’ll meet there !

See all 1 articles in conference

Patterns 1 articles

Now that I have a couple of minutes, let me tell you what I changed exactly in Mephisto to support multiple spam detection engines.

Initially, app/models/comment.rb looked like this:


1 class Comment < Content
2 def check_approval(site, request)
3 self.approved = site.approve_comments?
4 if valid_comment_system?(site)
5 akismet = Akismet.new(site.akismet_key, site.akismet_url)
6 self.approved = !akismet.comment_check(comment_spam_options(site, request))
7 logger.info "Checking Akismet (#{site.akismet_key}) for new comment on Article #{article_id}. #{approved? ? Approved : Blocked}"
8 logger.warn "Odd Akismet Response: #{akismet.last_response.inspect}" unless Akismet.normal_responses.include?(akismet.last_response)
9 end
10 end
11 end

app/models/comment.rb@bugfixing

The original method was intimately tied to Akismet: it would instantiate one, and use it directly. The first thing I needed to do was to be able to use any kind of spam detection engine, without knowing the details of it. In fact, what I needed was an instance of the Adapter pattern.

The code changed from the above to this:


1 class Comment < Content
2 def check_approval(site, request)
3 self.approved = site.approve_comments? || spam_engine(site).ham?(article.permalink_url(site, request), self)
4 end
5
6 def spam_engine(site)
7 site.spam_engine
8 end
9 end

app/models/comment.rb@multiengine

So now, I had to get hold of an instance of SpamDetectionEngine. I again turned to the Gang of Four patterns, and this is when I realized I needed a Strategy Pattern. The Site was the most obvious place to put this, as the configuration for storing Akismet details was already there. I simply extended the concept further to allow any engine to store any configuration settings in Site, and made Site return an instance of my spam engine:


1 class Site < ActiveRecord::Base
2 def spam_engine
3 klass_name = read_attribute(:spam_detection_engine)
4 return Mephisto::SpamDetectionEngines::NullEngine.new(self) if klass_name.blank?
5 klass_name.constantize.new(self)
6 end
7 end

app/models/site.rb@multiengine

What’s a NullEngine you ask ? It’s the default engine. It’s a do nothing engine. It’s an instance of the Null object pattern. What does this engine do really ? It returns canned data for all requests. It’s always #valid?, and it always accepts comments.

So, I moved the old Akismet code to the AkismetEngine, and created a new DefensioEngine. Both of these engines have real code in place to do the validation. Although I haven’t verified that the AkismetEngine still works, I believe it should. I used Marc-André Cournoyer’s Defensio Rails plugin to actually talk to Defensio.

Next, I needed a way to configure those different engines. What I needed was a way to render templates, and have the engines provide the templates to the regular Rails views. I made myself a custom Template class, which renders a simple ERB template. I put the templates right next to the engine it’s linked to. Then it was just a matter of rendering the template. Looking at the AkismetEngine, the #settings_template code looks like this:


1 module Mephisto
2 module SpamDetectionEngines
3 class AkismetEngine < Mephisto::SpamDetectionEngine::Base
4 class << self
5 def settings_template(site)
6 load_template(File.join(File.dirname(FILE), "akismet_settings.html.erb")).render(:site => site, :options => site.spam_engine_options)
7 end
8 end
9 end
10 end
11 end

lib/mephisto/spam_detection_engines/akismet_engine.rb@multiengine

Finally, Defensio provides statistics through it’s API. I wanted to show nice graphics, and turned to the Google Chart API to generate them. In the end, what does all of this look like ? Well, this:

The Defensio engine's configuration and statistics
Click for larger version

I had a bit of discussion with Carl Mercier about the accuracy graph. He said Defensio looked poor because the green bar is very low. I countered that the difference between 95 and 96 percent, if the graph were 100 pixels high, would only be 1 pixel. The graph’s scale is actually 95 to 100%, so now the difference between 95 and 96 percent will be 20 pixels, if the bar is 100 pixels high. Anyway, this is an unresolved issue, and I’d like to know what people feel. You can change this yourself by cloning the repository and editing the Defensio statistics template:


1 <%
2
spam = statistics.spam.to_i</span> <span class="no"> 3</span> </span> ham = <span class="iv">statistics.ham.to_i
4 accuracy = statistics</span>.accuracy.to_f * <span class="fl">100.0</span> <span class="no"> <strong>5</strong></span> mapped_accuracy = ((accuracy - <span class="i">95</span>) * <span class="fl">5.0</span>) <span class="no"> 6</span> spam_pct = spam / (spam + ham) * <span class="fl">100.0</span> <span class="no"> 7</span> ham_pct = <span class="fl">100.0</span> - spam_pct <span class="no"> 8</span> false_positives, false_negatives = <span class="iv">statistics.false_positives.to_i, @statistics.false_negatives.to_i
9 false_positives_pct = false_positives.to_f / (false_positives + false_negatives) * 100.0
10 false_negatives_pct = 100.0 – false_positives_pct
11 stats_chd = sprintf("t:%.1f,%.1f", spam_pct, ham_pct)
12 accuracy_chd = sprintf("t:%.1f|%.1f", mapped_accuracy, 100.0)
13 retraining_chd = sprintf("t:%.1f,%.1f", false_positives_pct, false_negatives_pct)
14 %>
15

lib/mephisto/spam_detection_engines/defensio_statistics.html.erb@multiengine

All in all, I really enjoyed refactoring a tool that I use regularly. I didn’t find nor fix any major problems along the way, but if I do find some, I’ll be sure to fix them and make them available.

So, what are you waiting for ?


1 git clone git://github.com/francois/mephisto.git mephisto_defensio
2 cd mephisto_defensio
3 git checkout multiengine
4 rake db:bootstrap db:migrate
5 thin start

NOTE: There is a book called Refactoring to Patterns. I haven’t read it yet, but it seems like a good read.

See all 1 articles in patterns

Contributions 1 articles

I just received 2 contributions from Josh Nichols:

  • New —repository-type option on piston import to force the repository backend to use (instead of letting Piston guess), and for cases where Piston is unable to guess: ea958dd;
  • Test suite reorganization: 1cef7b6 and 9cfa8f3

Both contributions were accepted and are now part of Piston’s master branch. Thank you very much, Josh, for your work.

If you want to help, do not fear !


1 $ git clone git://github.com/francois/piston.git
2 $ # make changes
3 $ git commit
4 $ # fork piston’s repository
5 $ git remote add github git@github.com:YOURNAME/piston.git
6 $ git push github master
7 $ # Send me a pull request

See all 1 articles in contributions

Musings 1 articles

While convalescing from a recent surgery, I had a flash of insight. One can view the Internet and a file system as one big, really big, Hash.

What is a Hash if not a way to access a value referenced by a key ?

What is a filename ? A key.

What is a URL ? A key.

the_internet.rb

1 require uri
2 require open-uri
3 require singleton
4
5 # Access the Internet through a Hash-like interface
6 # Code originally written by
7 # François Beausoleil (francois@teksol.info)
8 # This code is released in the public domain.
9 class TheInternet
10 include Singleton
11
12 def [](url)
13 uri = url.respond_to?(:read) ? url : URI.parse(url)
14 uri.read
15 end
16
17 def []=(url, value)
18 # POST, PUT or whatever to URL
19 end
20 end

filesystem.rb

1 require singleton
2
3 # Access the filesystem through a Hash-like interface
4 # Code originally written by
5 # François Beausoleil (francois@teksol.info)
6 # This code is released in the public domain.
7 class Filesystem
8 include Singleton
9
10 def [](path)
11 File.read(path)
12 end
13
14 def []=(path, value)
15 File.open(path, wb) do |f|
16 f.write(value)
17 # Could do fancy stuff here:
18 # YAMLize value, marshall it, etc.
19 end
20 end
21 end

The above allows us to do very interesting things:


1 # Copy from the Internet to a local file:
2 Filesystem.instance[teksol.info/index.html] =
3 TheInternet.instance[http://blog.teksol.info/]
4
5 # Given a suitable implementation of TheInternet#[]=,
6 # we could do:
7 search_results =
8 (TheInternet.instance[http://www.google.com/search] =
9 {:hl => en, :q => rails})

Is this rambling interesting ? Certainly. Is it useful ? You tell me !

DISCLAIMER: The Filesystem object above is full of security holes. It should be implemented in a chrooted, environment, check permissions, etc.

See all 1 articles in musings

Relationships 1 articles

Over on rails-core, I posted Edge Rails fails saving parent when has_many child ?.

The models I am using are:

app/models/invoice.rb

1 class Invoice < ActiveRecord::Base
2 belongs_to :customer
3 has_many :lines, :class_name => InvoiceItem
4 validates_presence_of :no, :customer
5 end

app/models/invoice_item.rb

1 class InvoiceItem < ActiveRecord::Base
2 belongs_to :invoice
3 validates_presence_of :invoice
4 end

As is, it was impossible to use the normal build and save idiom:


1 $ ruby script\console
2 Loading development environment.
3 >> invoice = Invoice.new
4 => #<Invoice:0×3a76060 …>
5 >> line = invoice.lines.build
6 => #<InvoiceItem:0×3a5d610 …>
7 >> invoice.save!
8 ActiveRecord::RecordInvalid: Validation failed: Lines is invalid
9 from
10 ./script/../config/../config/../vendor/rails/activerecord/lib/active_record/validations.rb:736:in
11 `save!’
12 from (irb):3
13 >> puts line.errors.full_messages
14 Invoice can’t be blank
15

Well, of course. I know invoice can’t be blank. If I remove :invoice from the validates_presence_of, things work out fine:


1
2 >> invoice.save!
3 => nil
4 >> invoice.new_record?
5 => false
6 >> line.new_record?
7 => false
8 >> line.invoice
9 => nil
10 >> # Huh?

Digging into the code, ActiveRecord::Associations::AssociationProxy#set_belongs_to_association_for it turns out that only the foreign key is assigned to the child instance, not the full parent model. The behavior as seen above is therefore “normal”.

Turns out that if I validate the foreign key instead, things work perfectly:

app/models/invoice_item.rb

1 class InvoiceItem < ActiveRecord::Base
2 belongs_to :invoice
3 validates_presence_of :invoice_id
4 end


1
2 >> invoice.save!
3 => nil
4 >> line.new_record?
5 => false
6 >> line.invoice
7 => #<Invoice:0×39fbea8 …>

Lesson learned: don’t validate the presence of the associated model, only it’s foreign key.

See all 1 articles in relationships

Mongodb 1 articles

In the past couple of weeks, I had the opportunity to look at MongoDB and Redis in more details. Both are fast, but serve different needs. Here’s my take on them both.

MongoDB

MongoDB is a document oriented database. MongoDB stores structured data, as documents:


1 {
2 name: François Beausoleil,
3 email: francois@teksol.info,
4 year-of-birth: 1973,
5 tags: [friendly, ruby, coder, father]
6 }

Since MongoDB stores structured data, it has a richer query language:


1 // Find people that are fathers OR coders
2 db.people.find( { "tags" : { $in : [father, coder] } } );
3
4 // Find people that are fathers AND coders
5 db.people.find( { "tags" : { $all : [father, coder] } } );
6
7 // Index the email member of documents
8 db.people.ensureIndex({email: 1});
9
10 // Query using the index
11 db.people.findOne({email: francois@teksol.info})

Redis

Redis is a key-value store that is strongly typed. Redis has three types of values: strings, sets and lists. Out of the box, you cannot store structured data in Redis. You have to build the index manually, storing yet other keys.

Redis is conceptually simple. All operations are either stores or fetches:


1 PUT key value
2 GET key

You can think of Redis as a persistent Hash or Dictionary. The fact that Redis is strongly typed makes some operations very interesting.


1 # Store other data as serialized JSON object, but we won’t be able to query the value itself
2 SET "people:francois" "{’name’: ‘François Beausoleil’, ‘email’: ‘francois@teksol.info’, ‘year-of-birth’: 1973, ‘tags’: [‘friendly’, ‘ruby’, ‘coder’, ‘father’]}"
3
4 # Build ourselves an index on the email value
5 SET "email:francois@teksol.info" "francois"
6
7 # Build another index for tags
8 SADD "tags:friendly" "francois"
9 SADD "tags:ruby" "francois"
10 SADD "tags:coder" "francois"
11 SADD "tags:father" "francois"
12
13 # During authentication, we would find the person using the email address
14 GET "email:francois@teksol.info"
15
16 # Get the rest of the data
17 GET "people:francois"
18
19 # Find the keys that have both father AND coder as tags
20 SINTER "tags:father" "tags:coder"

Conclusion

So, which one should you choose? The short answer is “it depends”.

The longer answer is to use the best tool for the job. Storing documents is different than storing simple key values. Of course, both can be used for the same job, but Redis will be easier if you need a simple Hash/Dictionary. Use MongoDB for storing structured data.

See all 1 articles in mongodb

Redis 1 articles

In the past couple of weeks, I had the opportunity to look at MongoDB and Redis in more details. Both are fast, but serve different needs. Here’s my take on them both.

MongoDB

MongoDB is a document oriented database. MongoDB stores structured data, as documents:


1 {
2 name: François Beausoleil,
3 email: francois@teksol.info,
4 year-of-birth: 1973,
5 tags: [friendly, ruby, coder, father]
6 }

Since MongoDB stores structured data, it has a richer query language:


1 // Find people that are fathers OR coders
2 db.people.find( { "tags" : { $in : [father, coder] } } );
3
4 // Find people that are fathers AND coders
5 db.people.find( { "tags" : { $all : [father, coder] } } );
6
7 // Index the email member of documents
8 db.people.ensureIndex({email: 1});
9
10 // Query using the index
11 db.people.findOne({email: francois@teksol.info})

Redis

Redis is a key-value store that is strongly typed. Redis has three types of values: strings, sets and lists. Out of the box, you cannot store structured data in Redis. You have to build the index manually, storing yet other keys.

Redis is conceptually simple. All operations are either stores or fetches:


1 PUT key value
2 GET key

You can think of Redis as a persistent Hash or Dictionary. The fact that Redis is strongly typed makes some operations very interesting.


1 # Store other data as serialized JSON object, but we won’t be able to query the value itself
2 SET "people:francois" "{’name’: ‘François Beausoleil’, ‘email’: ‘francois@teksol.info’, ‘year-of-birth’: 1973, ‘tags’: [‘friendly’, ‘ruby’, ‘coder’, ‘father’]}"
3
4 # Build ourselves an index on the email value
5 SET "email:francois@teksol.info" "francois"
6
7 # Build another index for tags
8 SADD "tags:friendly" "francois"
9 SADD "tags:ruby" "francois"
10 SADD "tags:coder" "francois"
11 SADD "tags:father" "francois"
12
13 # During authentication, we would find the person using the email address
14 GET "email:francois@teksol.info"
15
16 # Get the rest of the data
17 GET "people:francois"
18
19 # Find the keys that have both father AND coder as tags
20 SINTER "tags:father" "tags:coder"

Conclusion

So, which one should you choose? The short answer is “it depends”.

The longer answer is to use the best tool for the job. Storing documents is different than storing simple key values. Of course, both can be used for the same job, but Redis will be easier if you need a simple Hash/Dictionary. Use MongoDB for storing structured data.

See all 1 articles in redis

Web 1 articles

I’m posting this article from Firefox Beta 3. From a limited amount of testing, I feel like the speed is improved. Part of that might just be Firebug being disabled, but it’s nice anyway.

See all 1 articles in web

Rubycocoa 1 articles

Lately, I’ve been having fun with RubyCocoa while reading Brian Marick‘s RubyCocoa. It’s an entertaining book, and I really like RubyCocoa itself. But I hit a little gotcha. Take the following code:

PathController.rb

1 require osx/cocoa
2
3 class PathController < OSX::NSObject
4 ib_outlet :path, :events
5 ib_action :choosePath
6
7 def choosePath(sender)
8 panel = OSX::NSOpenPanel.openPanel
9 panel.canChooseFiles = false
10 panel.canChooseDirectories = true
11 panel.allowsMultipleSelection = false
12
13 result = panel.runModal
14 if result == OSX::NSFileHandlingPanelOKButton then
15 self.path.title = panel.filename
16 end
17 end
18 end

What could be wrong with this? Turns out #ib_oulet does not define accessors for the instance variables it defines. Look at the exception:


1 OSX::OCMessageSendException: Can’t get Objective-C method signature for selector ‘events’ of receiver #<PathController:0×2297be class=‘PathController’ id=0×534ec0>
2 /Library/Frameworks/RubyCocoa.framework/Resources/ruby/osx/objc/oc_wrapper.rb:50:in `ocm_send’
3 /Library/Frameworks/RubyCocoa.framework/Resources/ruby/osx/objc/oc_wrapper.rb:50:in `method_missing’
4 /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/PathController.rb:27:in `choosePath’
5 /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/rb_main.rb:22:in `NSApplicationMain’
6 /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/rb_main.rb:22
7 /Library/Frameworks/RubyCocoa.framework/Resources/ruby/osx/objc/oc_wrapper.rb:50:in `ocm_send’: Can’t get Objective-C method signature for selector ‘events’ of receiver #<PathController:0×2297be class=‘PathController’ id=0×534ec0> (OSX::OCMessageSendException)
8 from /Library/Frameworks/RubyCocoa.framework/Resources/ruby/osx/objc/oc_wrapper.rb:50:in `method_missing’
9 from /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/PathController.rb:27:in `choosePath’
10 from /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/rb_main.rb:22:in `NSApplicationMain’
11 from /Users/francois/Projects/fsevtest/build/Debug/fsevtest.app/Contents/Resources/rb_main.rb:22

From the obscure message, I gathered this was some kind of NoMethodError (why couldn’t they use an exception with a good name?) That was Sunday morning. Sunday evening, I suddenly had a flash that ib_outlet wouldn’t define an accessor. Changed my code to use the instance variable instead:

PathController.rb

1 if result == OSX::NSFileHandlingPanelOKButton then
2 @path.title = panel.filename
3 end

Lo and behold! It worked beautifully.

See all 1 articles in rubycocoa

Rubyfringe-2008 1 articles

Well, the first day is over. The talks were very interesting, especially Obie’s. Obie talked about how we should market ourselves / our company, and how to close deals. All of this essentially relates to a power balance between the lead / customer and us.

I was very impressed with the organization of the conference. One problem I had is the hackfest. We were in an art gallery, which is a very interesting place to be. Unfortunately, we didn’t have enough power outlets for everyone. And the place that was offered for hacking was in a cinema room. “Kill Bill, Vol. 1” was playing, which is interesting, but it distracted us. Then the space was pretty minimal to hold the laptop.

Anyways, I’m enjoying myself very much. I was supposed to work on Piston, which I have barely started on right now. That’s what I’m going to do next.

See all 1 articles in rubyfringe-2008

Hiring 1 articles

I would like to provide you with an interesting link: How to Get Hired — What CS Students Need to Know

When I was searching for someone myself, I hadn’t read this essay, but I already knew I was searching for someone who is Smart and Gets Things Done.

Those two articles pretty much sum up what I want from anyone I would be hiring.

See all 1 articles in hiring

Rais 1 articles

I recently hit upon the Enhanced Migrations plugin by Revolution on Rails. Works great when you develop on branches. The #dump_schema_information method of ActiveRecord::ConnectionAdapters::SchemaStatements only dumps the most recently migration file. Since each migration is now a separate entry in the migrations_info table, we can’t report only the latest one.

To this end, I generated the following diff:


1 $ svn diff vendor
2 Index: vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb
3 ===============
4 - vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb (revision 7767)
5 + vendor/plugins/enhanced_migrations/lib/enhanced_migrations.rb (working copy)
6 </span><span class="er"> -58,8 +58,8 </span><span class="er">
7
8 ActiveRecord::ConnectionAdapters::SchemaStatements.send(:define_method, :dump_schema_information) do
9 begin
10if (current_schema = ActiveRecord::Migrator.current_version) > 0
11return "INSERT INTO #{ActiveRecord::Migrator.schema_info_table_name} VALUES (#{current_schema}, NOW"
12 + select_all("SELECT * FROM #{ActiveRecord::Migrator.schema_info_table_name} ORDER BY created_at, id").map do |migration|
13 + "INSERT INTO #{ActiveRecord::Migrator.schema_info_table_name} VALUES;\n"
14 end
15 rescue ActiveRecord::StatementInvalid
16 # No Schema Info

Hope this is useful for other people.

See all 1 articles in rais

Sti 1 articles

Active Record has a nice feature that enables the programmer to separate concerns. In the application I am building, I need to reject some transactions based on business rules. For example, if an attacker attemps to empty an account by requesting multiple payouts, or attempting to load more than a specific amount of money in the user’s account.

In this application, I am using STI (Single Table Inheritance) to represent the different types of transaction. The hierarchy looks like this:


1 class GreenbackTransaction < ActiveRecord::Base
2 end
3
4 class AccountRechargeTransaction < GreenbackTransaction
5 end
6
7 class AccountPayoutTransaction < GreenbackTransaction
8 end

I defined my observer like this:

app/models/greenback_transaction_observer.rb

1 class GreenbackTransactionObserver < ActiveRecord::Observer
2 def after_create(txn)
3 # Code to reject the transaction if it is suspicious
4 end
5 end

Using the console, everything was working fine. So, off I went to write a test for it (I know, it should have been the other way around, but I haven’t used observers much, and I wanted to see what was going on).

My test is defined like this:

test/unit/greenback_transaction_observer_test.rb

1 class GreenbackTransactionObserverTest < Test::Unit::TestCase
2 fixtures :greenback_transactions, :users, :accounts, :affected_accounts
3
4 def setup
5 sam</span> = users(<span class="sy">:sam</span>) <span class="no"> 6</span> <span class="co">Setting</span>.recharge_amount_threshold = <span class="i">150</span>.to_money <span class="no"> 7</span> <span class="r">end</span> <span class="no"> 8</span> <span class="no"> 9</span> <span class="r">def</span> <span class="fu">test_suspicious_transaction_rejected</span> <span class="no"><strong>10</strong></span> assert_nothing_raised <span class="r">do</span> <span class="no">11</span> <span class="co">AccountRechargeTransaction</span>.new(<span class="sy">:account</span> =&gt; <span class="iv">sam.account, :amount => 100.to_money)
12 end
13
14 assert_raise(TransactionFailureException) do
15 AccountRechargeTransaction.new(:account => @sam.account, :amount => 100.to_money)
16 end
17 end
18 end

To my complete surprise, this didn’t work. After much investigation, I found that the observer was not loaded for GreenbackTransaction. After some fooling around, adding logging statements in Rails core, I finally stumbled upon the solution:

app/models/account_recharge_transaction_observer.rb

1 class AccountRechargeTransactionObserver < ActiveRecord::Observer
2 observe AccountRechargeTransaction
3
4 def after_create(txn)
5 # Code to reject the transaction if it is suspicious
6 end
7 end

The problem was the observer was registered on GreenbackTransaction, and it seems the observers aren’t inherited in subclasses. This is important and bears repeating: if you use STI and observers, observe your subclasses !.

See all 1 articles in sti

Sysadmin 1 articles

I sometimes have to do sysadmin work, such as when I’m the sole technical person on a probject. When I need to keep a service running, I usually turn to daemontools. Daemontools was written by D. J. Bernstein, a mathematician and author of many UNIX utilities.

From daemontools’ home page:

daemontools is a collection of tools for managing UNIX services.

daemontools home page

What this means is daemontools is designed to run, and keep running, a service. Daemontools also includes other utilities which I find useful, such as setuidgid, envdir and multilog,. I searched for an article such as this, but couldn’t find it. If you find a factual error, please let me know immediately. If you have your own best practices, let me know so I can expand on the list.

Read the articles themselves here:

See all 1 articles in sysadmin

Scgi 1 articles

It is unknown if this breakage still exists. I don’t use SCGI anymore (I use Phusion’s Passenger instead), so this information may be competely irrelevant.

The project I’m working on currently must protect access to some files. They are Flash applications which the user must be authenticated to use.

The best way to do that in Rails is to hide the files in a non-public directory, and then use send_file to stream the file to the browser.

Unfortunately, it seems that using SCGI and Apache breaks #send_file. The following errors are shown in Apache’s error log:

[Mon Dec 26 14:39:47 2005] [error]
[client 69.70.167.187]
malformed header from script. Bad header=FWS\x05\xbf\x97: binary
[Mon Dec 26 14:39:47 2005] [error]
[client 69.70.167.187]
(500)Unknown error: scgi: Unknown error: error reading response headers

(Content reformatted for readability)

I searched the Rails mailing list, and found SCGI error with send_data. No conclusive information in the thread, and Zed has not released another version of SCGI yet.

Just for the heck, I removed the default buffering that #send_file does:


1 send_file request_file, :disposition => inline,
2 :filename => @app.name,
3 :stream => false

Bingo ! Things started working perfectly fine after that.

See all 1 articles in scgi

Money 1 articles

Hongli Lai took the lead/moved Money to GitHub. And the bug didn’t exist a couple of days after I wrote this entry originally.

The Money library is a useful. Unfortunately, release 1.5.9 has an error. One cannot substract an amount from the empty Money.


1 def test_substract_from_zero
2 assert_equal -12.to_money, Money.empty – (12.to_money)
3 end

The test case will fail with this:


1 1) Failure:
2 test_substract_from_zero(MoneyTest) [./test/unit/money_test.rb:29]:
3 <#<Money:0×3accb68 @cents=-1200, @currency="USD">> expected but was
4 <#<Money:0×3acca60 @cents=1200, @currency="USD">>.

The error is caused by Money#-(other_money):


1 def -(other_money)
2 return other_money.dup if cents.zero?
3
4 end

return other_money.dup if cents.zero? should be removed to allow the test case to pass.

Update (2006-02-13): Tobias already posted a new version of the library, available as a gem.

See all 1 articles in money

Mor7 1 articles

If you weren’t at Montreal on Rails 7, you really should have been there. Not because of me, but because Marc-André showed how Thin integrates with Rack, and coded a web framework in 15 minutes. Then, Julien Guimont demoed his Encrypted URL Helper. Very interesting stuff, if you need to obfuscate URLs.

Then, there was me. A quick demo of Google Charts, and this post to provide you all with the link to the mor7-google-charts-demo Git repository:

git://github.com/francois/mor7-google-charts-demo.git

Go ahead and have fun. If you make significant changes, please contact me so I can pull your changes and push them back to the public repository.

See all 1 articles in mor7

Browser 1 articles

I’m posting this article from Firefox Beta 3. From a limited amount of testing, I feel like the speed is improved. Part of that might just be Firebug being disabled, but it’s nice anyway.

See all 1 articles in browser

Active resource 1 articles

I have a work-in-progress branch for using Digest authentication with ActiveResource.

So, how did I do it? It wasn’t too hard actually. When I spiked, I changed ActiveResource::Connection#request to handle authentication itself. I ended up with a big mess: a new rescue clause, 10 lines of code to calculate the digest and so on. But I knew it would work. So, I git checkout . and started with tests, as it should.

The way ActiveResource is built, if a username / password is sent in, ActiveResource will send those automatically in an Authorization header, using the Basic authentication method. I need a way to turn this off. Thus grew #use_basic_authentication= and #use_digest_authentication=.

Next up, actually being able to calculate the Digest. A quick search turned up code by Eric Hodel in the form of a Ruby module: An implementation of HTTP Digest Authentication in Ruby

After a bit of cleanup and rewriting, I have a branch of ActiveResource that’s ready to be commented on. Please see francois/ar_digest and leave comments there.

An example of using Digest would be:


1 require "logger"
2 require "activeresource"
3 require "pp"
4
5 ActiveResource::Base.logger = Logger.new(STDERR)
6
7 ActiveResource::Base.site = "http://adgear.local/api"
8 ActiveResource::Base.user = "francois"
9 ActiveResource::Base.password = "my-funny-new-password-which-you’ve-never-seen-before"
10 ActiveResource::Base.timeout = 30
11
12 # Don’t attempt Basic authentication, but be sure to use Digest
13 ActiveResource::Base.use_basic_authentication = false
14 ActiveResource::Base.use_digest_authentication = true
15
16 class Site < ActiveResource::Base
17 end
18
19 pp sites = Site.find(:all)

This work was sponsored by Bloom Digital Platforms, as part of my work on their AdGear API.

See all 1 articles in active_resource

Hosting 1 articles

I have the following CRON entries for one of my sites:


1 39 * * * * killall -9 dispatch.fcgi >/dev/null 2>&1
2 0,4,8,12,16,20,24,28,32,36,40,44,48,52,56 * * * * \
3 curl http://www.weputlightsup.com/ >/dev/null 2>&1

What does this do ? The first entry kills all of my FastCGI processes at 39 minutes past the hour. The second one pings a specific URL to ensure the FastCGI processes are always up and running, every four minutes.

I am certainly not the first to mention this technique, see View processes, grep out by user and then kill all their PIDs.

Seems strange though that I couldn’t find much references to this technique while Googling around.

Anyway, I hope this helps someone.

See all 1 articles in hosting

Tdd 1 articles

Today, I was working on my lawn. Spreading new soil to sow new grass. I was looking at the ground, and I was pretty sure drainage wasn’t going to be good. I decided to make a test: I got out the garden hose and let it flow. Sure enough, water was pooling instead of draining. Back to the drawing board (or at least, a couple more rakes to spread things around).

As I was working the soil, a thought hit me: I’d say nearly every profession has been doing test driven design (not development) since pretty much the dawn of time:

  • the toolsmith in the ancient tribes would hit the stone, see if it did work or not, and repeat as appropriate;
  • the chef tastes his food before letting other people eat it;
  • the plumber tests the pipes before letting the water flow in;
  • the soldier tests his equipment before setting off on the battlefield;
  • naval engineers are simulation testing their latest aircraft carrier in an effort to ensure nothing is out of place;
  • certain programmers write automated tests for their code, to ensure it works as designed.

If nearly every profession has been doing it for thousands of years, why aren’t you doing it today?

See all 1 articles in tdd

Key-value-store 1 articles

In the past couple of weeks, I had the opportunity to look at MongoDB and Redis in more details. Both are fast, but serve different needs. Here’s my take on them both.

MongoDB

MongoDB is a document oriented database. MongoDB stores structured data, as documents:


1 {
2 name: François Beausoleil,
3 email: francois@teksol.info,
4 year-of-birth: 1973,
5 tags: [friendly, ruby, coder, father]
6 }

Since MongoDB stores structured data, it has a richer query language:


1 // Find people that are fathers OR coders
2 db.people.find( { "tags" : { $in : [father, coder] } } );
3
4 // Find people that are fathers AND coders
5 db.people.find( { "tags" : { $all : [father, coder] } } );
6
7 // Index the email member of documents
8 db.people.ensureIndex({email: 1});
9
10 // Query using the index
11 db.people.findOne({email: francois@teksol.info})

Redis

Redis is a key-value store that is strongly typed. Redis has three types of values: strings, sets and lists. Out of the box, you cannot store structured data in Redis. You have to build the index manually, storing yet other keys.

Redis is conceptually simple. All operations are either stores or fetches:


1 PUT key value
2 GET key

You can think of Redis as a persistent Hash or Dictionary. The fact that Redis is strongly typed makes some operations very interesting.


1 # Store other data as serialized JSON object, but we won’t be able to query the value itself
2 SET "people:francois" "{’name’: ‘François Beausoleil’, ‘email’: ‘francois@teksol.info’, ‘year-of-birth’: 1973, ‘tags’: [‘friendly’, ‘ruby’, ‘coder’, ‘father’]}"
3
4 # Build ourselves an index on the email value
5 SET "email:francois@teksol.info" "francois"
6
7 # Build another index for tags
8 SADD "tags:friendly" "francois"
9 SADD "tags:ruby" "francois"
10 SADD "tags:coder" "francois"
11 SADD "tags:father" "francois"
12
13 # During authentication, we would find the person using the email address
14 GET "email:francois@teksol.info"
15
16 # Get the rest of the data
17 GET "people:francois"
18
19 # Find the keys that have both father AND coder as tags
20 SINTER "tags:father" "tags:coder"

Conclusion

So, which one should you choose? The short answer is “it depends”.

The longer answer is to use the best tool for the job. Storing documents is different than storing simple key values. Of course, both can be used for the same job, but Redis will be easier if you need a simple Hash/Dictionary. Use MongoDB for storing structured data.

See all 1 articles in key-value-store

Exceptions 1 articles

In Ruby, methods automatically define begin/end blocks. That makes it easy to write exception handlers and ensure blocks:


1 def access!(options={})
2 raise AuthorizationFailure if self.email != options[:email]
3 self.download_count += 1
4
5 rescue AuthorizationFailure
6 self.unauthorized_access_count += 1
7 raise
8
9 ensure
10 self.save!
11 end

It is important to re-raise the exception in the exception handler, or else the exception will be silently thrown away.

See all 1 articles in exceptions

Sequel 1 articles

I ran into a little gotcha today, using Sequel. I’m writing an importer, you know the kind: read record from database A, apply some transformations, write to database B. No rocket science required. But, Sequel has a little gotcha that stumped me for a bit. My script looked like this:


1 DBa = Sequel.connect ""
2 DBb = Sequel.connect ""
3
4 class APerson < Sequel::Model(DBa[:people])
5 end
6
7 class BContact < Sequel::Model(DBb[:contacts])
8 end
9
10 contacts = Hash.new
11 APerson.all.each do |person|
12 contact = BContact.create(
13 :name => person.last_name + ", " + person.first_name,
14 :tenant_code => ENV["TENANT_CODE"],
15 :updated_by => "import",
16 :updated_at => Time.now)
17 contacts[ person.id ] = contact.contact_id
18 end
19
20 # Now I can map A’s IDs to the correct value in database B, such as
21 # for attaching email addresses, phone numbers, etc.

The Contact model in the B database is declared like this:


1 create_table :contacts do
2 column :tenant_code, :integer, :null => false
3 column :contact_id, :serial, :null => false
4 column :name, "varchar(240)", :null => false
5
6 primary_key [:tenant_code, :contact_id]
7 foreign_key [:tenant_code], :tenants
8 end

Notice tenant_code and contact_id are part of the primary key. I don’t write to contact_id because I want the sequence’s value to be returned to me. But I must write my own value to the tenant_code column. I was receiving an exception on the #create call:


1 /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:1491:in `block in set_restricted’: method tenant_code= doesn’t exist or access is restricted to it (Sequel::Error)
2 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:1486:in
`
each
3 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:1486:in `set_restricted

4 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:1077:in `set’
5 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:1456:in
`
initialize_set
6 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:764:in `initialize

7 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:134:in `new’
8 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:134:in
`
create
9 from /Users/francois/.rvm/gems/ruby-1.9.2-p180/gems/sequel-3.25.0/lib/sequel/model/base.rb:248:in `find_or_create

10 from script/import:65:in `block (2 levels) in <top (required)>’
11

I was very much stumped, and turned to the excellent documentation. I eventually found my way to #set_all, and changed my code accordingly:


1 APerson.all.each do |person|
2 contact = BContact.new.set_all(
3 :name => person.last_name + ", " + person.first_name,
4 :tenant_code => ENV["TENANT_CODE"],
5 :last_updated_by => "import",
6 :last_updated_at => Time.now)
7 contacts[ person.id ] = contact.contact_id
8 end

Even though the Sequel RDoc says #set_all ignores restricted columns, I was still receiving the same exception. I was now doubly stumped, until I found a reference to #unrestrict_primary_key. I added the declaration to BContact and was able to proceed:


1 class BContact < Sequel::Model(DBb[:contacts])
2 unrestrict_primary_key
3 end

You know the drill though: where you import one model, you’ll have more to import shortly. Ruby to the rescue!


1 class Sequel::Model
2 # Open classes win every time!
3 unrestrict_primary_key
4 end

Problem solved!

See all 1 articles in sequel

Statistics 1 articles

Atomic Coding

2007-02-22

I just found out about Atomic Coding with Subversion and thought I’d point it out to you.

I ran TAW’s svn_log_summary, and here are my results:


1 Activity of francois:
2 Time between commits distribution:
3 * 10% – 28.205291s
4 * 25% – 51.901505s
5 * median – 5m12s
6 * 75% – 38m38s
7 * 90% – 11h 42m30s
8
9 Median number of affected files: 2
10 * 1 file – 81 times
11 * 2 files – 47 times
12 * 3 files – 20 times
13 * 4 files – 11 times
14 * 5 files – 4 times
15 * 6 files – 5 times
16 * 8 files – 2 times
17 * 10 files – 2 times
18 * 13 files – 1 time
19 * 18 files – 1 time
20 * 20 files – 1 time
21 * 38 files – 1 time
22 * 56 files – 1 time

Five minutes between commits ? Why, I didn’t know I was committing that often.

See all 1 articles in statistics

Apple 1 articles

Just received an email from Apple today:

From: devbugs@apple.com
Subject: Re: Bug ID 6543251: Git is not available by default on Leopard

Hello François,

This is a follow up to Bug ID# 6543251. After further investigation it has been determined that this is a known issue, which is currently being investigated by engineering. This issue has been filed in our bug database under the original Bug ID# 5404556. The original bug number being used to track this duplicate issue can be found in the State column, in this format: Duplicate/OrigBug#.

Thank you for submitting this bug report. We truly appreciate your assistance in helping us discover and isolate bugs.

Best Regards,

Kit Cheung
Apple Developer Connection
Worldwide Developer Relations

This is a followup to Want Git preinstalled on next Mac OS X?

See all 1 articles in apple

Fastcgi 1 articles

I have the following CRON entries for one of my sites:


1 39 * * * * killall -9 dispatch.fcgi >/dev/null 2>&1
2 0,4,8,12,16,20,24,28,32,36,40,44,48,52,56 * * * * \
3 curl http://www.weputlightsup.com/ >/dev/null 2>&1

What does this do ? The first entry kills all of my FastCGI processes at 39 minutes past the hour. The second one pings a specific URL to ensure the FastCGI processes are always up and running, every four minutes.

I am certainly not the first to mention this technique, see View processes, grep out by user and then kill all their PIDs.

Seems strange though that I couldn’t find much references to this technique while Googling around.

Anyway, I hope this helps someone.

See all 1 articles in fastcgi

Cron 1 articles

I have the following CRON entries for one of my sites:


1 39 * * * * killall -9 dispatch.fcgi >/dev/null 2>&1
2 0,4,8,12,16,20,24,28,32,36,40,44,48,52,56 * * * * \
3 curl http://www.weputlightsup.com/ >/dev/null 2>&1

What does this do ? The first entry kills all of my FastCGI processes at 39 minutes past the hour. The second one pings a specific URL to ensure the FastCGI processes are always up and running, every four minutes.

I am certainly not the first to mention this technique, see View processes, grep out by user and then kill all their PIDs.

Seems strange though that I couldn’t find much references to this technique while Googling around.

Anyway, I hope this helps someone.

See all 1 articles in cron

Helpers 1 articles

This article is obsolete. The changes have been rolled into recent versions of Rails such that calling:

1 <%= render :partial => "partial", :object => the_object %>
2

works.

I was reading the code for the FileColumn, helper and I had a thought. What if we could obviate the need for requiring instance variables for helpers ?

A recent example on the mailing list ? render partial and passing in the object, by Mark Van Holstyn. In the thread, he asks why he can’t do:

app/controllers/user_controller.rb

1 def edit
2 user</span> = <span class="co">User</span>.find_by_id(<span class="i">1</span>) <span class="no">3</span> render <span class="sy">:partial</span> =&gt; <span class="s"><span class="dl">'</span><span class="k">address</span><span class="dl">'</span></span>, <span class="sy">:object</span> =&gt; <span class="iv">user.address}
4 end

His partial being:

app/views/shared/_address.rhtml

1 ><%= text_area(‘address’, ‘street’) %>
2

Reading FileColumn helper’s code, and the code in Rails own helpers, the following idiom is used to get a hold of the instance variable:

file-column-0.3/lib/file_column_helper.rb

1 def url_for_file_column(object_name, method, suffix=nil)
2 object = instance_variable_get("@#{object_name.to_s}")
3
4 end

What if we did it this way instead ?

file-column-0.3x/lib/file_column_helper.rb

1 def url_for_file_column(object_name, method, suffix=nil)
2 object = case object_name
3 when String, Symbol
4 instance_variable_get("@#{object_name.to_s}")
5 else
6 object_name
7 end
8
9 end

Actually, this should all be abstracted away in some kind of helper for helpers. Let me take a moment to prepare a patch. I think this would make things quite a bit easier for partial users.

See all 1 articles in helpers

Console 1 articles

Rails was patched to prevent that error.

If you are like me, you develop your applications on Windows and deploy to Linux/Unix/FreeBSD, whatever.

On Rails 1.1, the console does not start anymore on Windows. It reports the following error:


1 $ ruby script\console
2 Loading development environment.
3 c:/ruby/lib/ruby/1.8/irb/init.rb:151:in `parse_opts’: undefined method `upcase’ for nil:NilClass (NoMethodError)
4 from c:/ruby/lib/ruby/1.8/irb/init.rb:19:in `setup’
5 from c:/ruby/lib/ruby/1.8/irb.rb:54:in `start’
6 from c:/ruby/bin/irb:13

There are two ways to fix this issue:

  1. Upgrade Ruby to 1.8.4, or
  2. Patch your vendor/rails/railties/lib/commands/console.rb to remove the “—prompt-mode single” option:



    1 Index: vendor/rails/railties/lib/commands/console.rb
    2 ===============
    3 —- vendor/rails/railties/lib/commands/console.rb (revision 4097)
    4 + vendor/rails/railties/lib/commands/console.rb (working copy)
    5 @</span> -22,4 +22,4 <span class="chg">@
    6 else
    7 puts "Loading #{ENV[‘RAILS_ENV’]} environment."
    8 end
    9 -exec "#{options[:irb]} #{libs} —prompt-mode simple"
    10 +exec "#{options[:irb]} #{libs}"




    1 Index: vendor/rails/railties/lib/commands/console.rb
    2 ===
    3 —- vendor/rails/railties/lib/commands/console.rb (revision 4097)
    4 vendor/rails/railties/lib/commands/console.rb (working copy)
    5 @&lt;/span&gt; -22,4 +22,4 &lt;span class="chg"&gt;@
    6 else
    7 puts "Loading #{ENV[‘RAILS_ENV’]} environment."
    8 end
    9 -exec "#{options[:irb]} #{libs} —prompt-mode simple"
    10 exec "#{options[:irb]} #{libs}"

UPDATE (2006-03-30): Ah, I just found the reason why irb chokes. Richard Livsey’s script/console with edge Rails on Windows with 1.8.2

See all 1 articles in console

Ci 1 articles

I’m just setting up Continuous Integration for all of our branches on XLsuite.com. Doing a search for “cruisecontrol.rb” on Google unearthed: Ci For The Web 2.0 Guy Or Gal. I’m running the steps right now on my RimuHosting VPS.

Things are progressing along quite rapidly. I’ll say more when I’m done.

See all 1 articles in ci

Cookies 1 articles

One thing’s for sure, it’s not obvious what’s going on when a test fails because of cookies. For that reason, I shied away from using cookies significantly.

Anyway, here’s a failure I was getting:



1 1) Failure:
2 test_create_session_with_right_username_and_password_and_remember_me(CreateSessionsControllerTest)
3 [test/functional/sessions_controller_test.rb:57:in `test_create_session_with_right_username_and_password_and_remember_me’
4 /home/francois/src/config/../vendor/plugins/mocha/lib/mocha/test_case_adapter.rb:19:in `run’]:
5 No :auth_token cookie in the response
6 -
7 auth_token:
8


1 1) Failure:
2 test_create_session_with_right_username_and_password_and_remember_me(CreateSessionsControllerTest)
3 [test/functional/sessions_controller_test.rb:57:in `test_create_session_with_right_username_and_password_and_remember_me’
4 /home/francois/src/config/../vendor/plugins/mocha/lib/mocha/test_case_adapter.rb:19:in `run’]:
5 No :auth_token cookie in the response
6 -
7 auth_token:
8 – cookie value


Turns out the @response.cookies Hash is not indifferent. You really have to use a String to get to the content:


1 def test_cookie_set
2 assert_equal ["some value"], response</span>.cookies[<span class="s"><span class="dl">&quot;</span><span class="k">auth_token</span><span class="dl">&quot;</span></span>], <span class="no">3</span> <span class="s"><span class="dl">&quot;</span><span class="k">this will succeed</span><span class="dl">&quot;</span></span> <span class="no">4</span> assert_equal [<span class="s"><span class="dl">&quot;</span><span class="k">some value</span><span class="dl">&quot;</span></span>], <span class="iv">response.cookies[:auth_token],
5 "this will always fail"
6 end

See all 1 articles in cookies

Development 1 articles

Well, it is started. If you want to follow Piston 2.0’s progress, head on over to the Piston GitHub Repository.

If you want, you can register to Piston’s Recent Commits on master.

What have I got so far ?

piston/commands/import.rb

1 require "piston/commands/base"
2
3 module Piston
4 module Commands
5 class Import < Piston::Commands::Base
6 attr_reader :options
7
8 def initialize(options={})
9 @options = options
10 logger.debug {"Import with options: #{options.inspect}"}
11 end
12
13 def run(revision, working_copy)
14 tmpdir = working_copy.path.parent + ".#{working_copy.path.basename}.tmp"
15
16 begin
17 debug {"Creating temporary directory: #{tmpdir}"}
18 tmpdir.mkdir
19 revision.checkout_to(tmpdir)
20 working_copy.create
21 working_copy.copy_from(tmpdir)
22 working_copy.remember(revision.remember_values)
23 working_copy.finalize
24 ensure
25 debug {"Removing temporary directory: #{tmpdir}"}
26 tmpdir.rmtree
27 end
28 end
29 end
30 end
31 end

This is the new import command. Everything is expressed in terms of high-level operations, and the different backends will take care of doing the right thing. In the case of Subversion, checking out the repository will involve running “svn checkout”. For Git, this will be “git clone”, and so on.

#remember in the context above is for storing the properties Piston requires. These are the old piston:root, piston:uuid Subversion properties. The different working copy backends will take care of storing this in a format that is suitable: Subversion properties when available, YAML file otherwise.

Also, Piston will be more like Merb: it will be split in multiple Gems. Piston, the main gem, will install piston-core as well as all backends. piston-svn and piston-git are the first two backends I am planning on adding.

I’m having fun reimplementing Piston like this ! No fun == no new features. Fun == many new features.

See all 1 articles in development

Dreamhost 1 articles

DreamHost had a teensy little problem with their billing system. Specifically, Josh Jones ran his biller for a couple of days in 2008, when it should have been 2007. The biller dutifully ran along and billed all the customers that hadn’t been billed yet as of December 2008.

That’s OK, I can understand things like that happen. But what I like best is this:

If this/these erroneous charge(s) by us resulted in you having any sort of overdraft/bounced check/nsf fee from your financial institution, please contact our support team from the web panel. … When we get this, we will put money on your credit card equal to the amount your bank charged you, as well as give you a DreamHost account credit for the same amount on top of that.

Josh Jones in The Aftermath

Not only will they refund the charges, but they will also credit your account for the same amount. They didn’t have to do that, but it’s nice of them to do so.

I use DreamHost for mail and file hosting / backups. I don’t use them anymore to deploy Rails applications. But this is good service.

See all 1 articles in dreamhost

Pattern 1 articles

The more I think about it, the more I believe Commands and Presenters are intertwined. If you don’t know what a Presenter is, I suggest reading these articles:

I’m building a gem to abstract the Command pattern in your applications. It’s not Rails-specific, but does know about ActiveRecord. Anyway, as I was writing my sample application, I noticed some duplication:

app/controllers/invitations_controller.rb

1 class InvitationsController < ApplicationController
2 def show
3 user</span> = <span class="co">User</span>.invited.with_token(params[<span class="sy">:id</span>]).first <span class="no">4</span> raise <span class="co">ActiveRecord</span>::<span class="co">RecordNotFound</span> <span class="r">unless</span> <span class="iv">user
5 render :action => :confirm
6 end
7 end

app/commands/confirm_invitation_request_command.rb

1 class ConfirmInvitationRequestCommand
2 Komando.make_command self
3
4 mandatory_steps do
5 user</span> = <span class="co">User</span>.invited.with_token(params[<span class="sy">:id</span>]).first <span class="no"> 6</span> raise <span class="co">ActiveRecord</span>::<span class="co">RecordNotFound</span> <span class="r">unless</span> <span class="iv">user
7
8 user</span>.activate!(<span class="iv">attributes)
9 end
10 end

Notice the first two lines of InvitationsController#show and ConfirmInvitationRequestCommand#mandatory_steps: identical. Then I thought, what if the Command was also a Presenter? Then, I could refactor appropriately:

app/commands/confirm_invitation_request_command.rb

1 class ConfirmInvitationRequestCommand
2 Komando.make_command self
3
4 def user
5 user</span> ||= <span class="r">begin</span> <span class="no"> 6</span> <span class="co">User</span>.invited.with_token(params[<span class="sy">:id</span>]).first.tap <span class="r">do</span> |user| <span class="no"> 7</span> raise <span class="co">ActiveRecord</span>::<span class="co">RecordNotFound</span> <span class="r">unless</span> user <span class="no"> 8</span> <span class="r">end</span> <span class="no"> 9</span> <span class="r">end</span> <span class="no"><strong>10</strong></span> <span class="r">end</span> <span class="no">11</span> <span class="no">12</span> mandatory_steps <span class="r">do</span> <span class="no">13</span> <span class="iv">user = User.invited.with_token(params[:id]).first
14 raise ActiveRecord::RecordNotFound unless user</span> <span class="no"><strong>15</strong></span> <span class="no">16</span> <span class="iv">user.activate!(@attributes)
17 end
18 end

app/controllers/invitations_controller.rb

1 class InvitationsController < ApplicationController
2 def show
3 @user = ConfirmInvitationRequestCommand.new(:token => params[:id]).user
4 render :action => :confirm
5 end
6 end

Notice how the business logic of finding a user by token is nicely tucked away.

Of course, while writing this post, I realized that finding a User by token should live in my model, not in the controller or the command, but still: bear with me.

The Presenter pattern makes some decisions for the view: how to get at the objects to be shown, how to arrange them, how to save them as a group. Turns out my Command does essentially the same thing. Maybe we can think of Presenter as a special-case of Command? Such that Presenter is a Command to view an object? That seems logical to me.

See all 1 articles in pattern

Error 1 articles


1 $ svn commit
2 svn: MKACTIVITY of ‘/!svn/act/313eea44-b740-40fe-8f1e-f02955f9868f’:
3 SSL negotiation failed: SSL error: decryption failed or bad record mac (https://svn.xlsuite.org)

There exists When performing Subversion operations involving a lot of data over SSL, I get the error SSL negotiation failed: SSL error: decryption failed or bad record mac on the Subversion FAQ. But I searched for that after the fact.

Turns out the error was related to a bad password. Talk about misleading…

See all 1 articles in error

Metrics 1 articles

Sometimes, you need to know what your program’s doing, or how long it’s taking to do something. You could always log to a file, then use a combination of grep, awk and/or wc to gather the statistics yourself, but why bother? There are many tools out there which will do exactly what you want, just use them: Cacti, Graphite or plain-old RRD.

For instance, at Yatter, we need to know how fast our ranking algorithms are running, and we must know how long the ranking takes compared to the number of users and pages we have on hand. Graphing is the perfect solution for that, and Graphite fit the bill just fine for us. But Graphite alone won’t do all that we need: we also needed a way to instrument our code, hence the Counters library:


1 require "counters"
2 require "sequel"
3
4 DB = Sequel.connect "jdbc:postgres://127.0.0.1:5432/db"
5 Counter = Counters::StatsD.new(:url => "udp://127.0.0.1:8125", :namespace => "ranker")
6
7 users = Counter.latency "fetch.users" do
8 DB[:users].all
9 end
10
11 pages = Counter.latency "fetch.pages" do
12 DB[:pages].all
13 end
14
15 Counter.magnitude "count.users", users.length
16 Counter.magnitude "count.pages", pages.length
17
18 Counter.latency "ranking" do
19 entropy = 1.0
20 while entropy > MIN_ENTROPY
21 Counter.hit "iteration"
22 # Reduce entropy
23 end
24 end

At the end of the day, we’ll have hierarchical counters in Graphite which will give us all kinds of statistics. From the API above, you can gather that values are stored under hierarchical keys separated by fullstops. If you’re interested in the code, make yourself at home with the Counters GitHub repository.

Counters is certified to run on JRuby 1.6.0 in 1.8 mode, and MRI 1.9.2.

See all 1 articles in metrics

Authentication 1 articles

The application’s models

We’ll need a Todo, TodoUser and TodoUserDatabase. Each user will keep a copy of it’s todos, and the database will give us methods to find and register new users. Let’s start with the user’s database:


1 Object subclass: #TodoUserDatabase
2 instanceVariableNames: ‘database’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-Model’

In case you never noticed, this is a message named #subclass:instanceVariableNames:classVariableNames:poolDictionaries:category, and the receiver is Object. Through the magic of code formatting, this looks like a class declaration, but it’s just another message.

The database object needs a way to initialize itself. We have two ways to do it: either at instance initialization time, or on first access time. Let’s go the latter way, which is what seems most prevalent in Smalltalk / Seaside code I’ve read:


1 database
2 ^ database ifNil: [database := OrderedCollection new]

Put this method in protocol private. Then, we need a way to add and remove users:


1 addUser: aUser
2 (self findWithLogin: aUser login)
3 ifNil: [self database add: aUser. ^ aUser]
4 ifNotNil: [self raiseDuplicateLoginName]
5
6 removeUser: aUser
7 database remove: aUser

These go in protocol accessing. Notice #addUser: calls #raiseDuplicateLoginName. Let’s define that immediately:


1 rraiseDuplicateLoginName
2 Error raiseSignal: ’Can’’t have two users with the same login’

Put this method in protocol error handling. #addUser: calls another helper method: #findWithLogin:. The implementation looks like this:


1 findWithLogin: aLogin
2 ^ self database detect: [:each | each login = aLogin] ifNone: [nil]

Again, put this method in protocol accessing. We’re done with the database side of things. We can now switch to the user itself.

TodoUser model


1 Object subclass: #TodoUser
2 instanceVariableNames: ‘login password todos’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-Model’

As the class comment, enter this text:

Instances of myself represent a user with a login and password, as well as a collection of Todo instances.

In protocol accessing, we define basic accessor methods:


1 login
2 ^ login
3
4 login: anObject
5 login := anObject
6
7 isSamePassword: aPassword
8 ^ password = aPassword
9
10 password: anObject
11 password := anObject
12
13 todos
14 ^ todos ifNil: [todos := OrderedCollection new]

Again, note how the todos instance variable is initialized if it wasn’t previously initialized. Equivalent Ruby code uses the ||= operator.

Then, we need to add and remove todos from the user:


1 addTodo: aTodo
2 self todos add: aTodo
3
4 removeTodo: aTodo
5 self todos remove: aTodo

Pretty simple, as things go. Put these in the accessing protocol. The final model is the Todo itself.

Todo model

Let’s declare the class:


1 Object subclass: #Todo
2 instanceVariableNames: ‘createdAt completed description completedAt’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-Model’

And the class’ comment:

Instances of myself represent a task that should be done, a todo. Todos are pretty simple: they have a description and a flag that identifies the completion status. Todos also keep track of when they were instantiated and completed.

This time around, we need an #initialize method:


1 initialize
2 createdAt := DateAndTime now

Put this in the initialization protocol. Next, in protocol accessing, we add a couple of basic accessors:


1 createdAt
2 ^ createdAt
3
4 description
5 ^ description
6
7 description: aString
8 description := aString
9
10 isCompleted
11 ^ completedAt notNil
12
13 completedAt
14 ^ completedAt
15
16 completedAt: aDateTime
17 completedAt := aDateTime
18
19 markCompleted
20 completedAt := DateAndTime now.

The Seaside UI: controllers and views

To kick things off, I define a new Seaside WAComponent subclass which will be our root component:


1 WAComponent subclass: #TodoComponent
2 instanceVariableNames: ‘’
3 classVariableNames: ’UserDatabase’
4 poolDictionaries: ‘’
5 category: ’Todo-UI-Seaside’

We define ourselves a class variable named UserDatabase which will hold an instance of TodoUserDatabase. Let’s give ourselves two accessor methods to the database: one class side and the other instance side. Put this one in accessing, class side:


1 userDatabase
2 ^ UserDatabase ifNil: [UserDatabase := TodoUserDatabase new]

Again, we see the same pattern: initialize unless already initialized. Back on the instance side, add this method in the accessing protocol:


1 userDatabase
2 ^ self class userDatabase

This is a simple call to the class side version of the method with the same name.

TodoAuthDecorator

To implement authentication, we must wrap the application within a decorator that will take care of these details for it. The decorator’s job is simple: authenticate or register a new user, and when authenticated, show the application instead of the authentication / registration form. I based this implementation on Seaside’s WABasicAuthentication, but mine uses a form instead of the HTTP Basic Access Authentication method. I could have subclassed WABasicAuthentication, but I wanted to learn how to do it manually before I reused code.

Seaside’s decorators have an owner, which is the decorated component. The decorator has a chance to let the component render itself or not, which is what happens later in #renderContentOn:.

Let’s start by declaring the class:


1 WADecoration subclass: #TodoAuthDecorator
2 instanceVariableNames: ‘login password passwordConfirmation authenticated authenticationMessage registrationMessage’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-UI-Seaside’

login, password and passwordConfirmation are used to hold the login and password during registration and authentication. I use authenticated as a simple boolean to determine if authentication was successful or not. authenticationMessage and registrationMessage are messages that will be shown to the user (think of Rail’s flash). I begin in the initialization protocol with:


1 initialize
2 authenticated := false

All Seaside decorators that want a chance to render around their owner must provide a #renderContentOn: method. This goes in the rendering protocol:


1 renderContentOn: html
2 authenticated
3 ifTrue: [self renderOwnerOn: html]
4 ifFalse: [self renderAuthenticationFormOn: html]

When authenticated, we render our owner (the decorated component), else we call #renderAuthenticationFormOn:, which looks like this:


1 renderAuthenticationFormOn: html
2 html heading: ‘Todo List’.
3 (html div) id: ‘auth’; with: [
4 (html div) class: ‘column returning’; with: [
5 self renderReturningUserFormOn: html].
6 (html div) class: ‘column new’; with: [
7 self renderNewUserFormOn: html]].
8 html div class: ‘clear’; with: [html space]

Most of this is setting up nested DIVs to create a two-column layout. Styling is handled by the #style method:


1 style
2 ^ ’
3 #auth-area a#logout { float: right}
4 #auth .column { width: 49%; float: left; }
5 #auth form label, #auth form input { display: block; }
6 .clear { clear: both; }
7

#renderAuthenticationFormOn: uses a couple of helper methods:


1 renderNewUserFormOn: html
2 html form: [
3 html heading: ‘I am a new user’ level: 2.
4 registrationMessage ifNotNilDo: [:msg|
5 html heading: msg level: 3].
6 html div: [
7 (html label) for: ‘new-login’; with: ‘Login’.
8 (html textInput) id: ‘new-login’; on: #login of: self].
9 html div: [
10 (html label) for: ‘new-password’; with: ‘Password’.
11 (html passwordInput) id: ‘new-password’; on: #password of: self].
12 html div: [
13 (html label) for: ‘new-password-confirmation’; with: ‘Confirm password’.
14 (html passwordInput) id: ‘new-password-confirmation’; on: #passwordConfirmation of: self].
15 (html submitButton) on: #register of: self]
16
17 renderReturningUserFormOn: html
18 html heading: ‘I am a returning user’ level: 2.
19 authenticationMessage ifNotNilDo: [:msg|
20 html heading: msg level: 3].
21 html form: [
22 html div: [
23 (html label) for: ‘login’; with: ‘Login’.
24 (html textInput) id: ‘login’; on: #login of: self].
25 html div: [
26 (html label) for: ‘password’; with: ‘Password’.
27 (html passwordInput) id: ‘password’; on: #password of: self].
28 (html submitButton) on: #authenticate of: self]

WATag’s #on:of: message is pretty powerful: it sends the specified message to the specified object. WATextInputTag also sends a mutator message to the object when doing form submissions. The returning user case calls #authenticate on self, which is implemented as follows:


1 authenticate
2 | user |
3 user := self userDatabase findWithLogin: login.
4 user ifNil: [
5 self failAuthentication: ‘Unable to authenticate using these credentials.’]
6 ifNotNil: [
7 (user isSamePassword: password)
8 ifTrue: [
9 authenticationMessage := nil.
10 self owner user: user.
11 authenticated := true]
12 ifFalse: [
13 self failAuthentication: ‘Invalid credentials for user.’]]

This goes in the actions protocol. When registering, we instead call #register:


1 register
2 | user |
3 user := self userDatabase findWithLogin: login.
4 user
5 ifNotNil: [
6 self failRegistration: ‘Login already taken’]
7 ifNil: [
8 password = passwordConfirmation
9 ifTrue: [
10 authenticationMessage := nil.
11 user := TodoUser new.
12 user login: login; password: password.
13 self userDatabase addUser: user.
14 self authenticate]
15 ifFalse: [
16 self failRegistration: ‘Password and confirmation do not match’]]

If the user doesn’t already exist (as identified through the login), and the password and the confirmation match, we register the new user and immediately authenticate him. Once the user is authenticated, we finally let the decorated component (the decorator’s owner) render itself:


1 renderOwnerOn: html
2 (html div) id: ‘auth-area’; with: [
3 (html anchor) id: ‘logout’; on: #logout of: self.
4 html heading: login capitalized, ’’’s Todo List’].
5 super renderOwnerOn: html

Put this in the rendering protocol. Here, we provide a logout link for authenticated users, as well as show who’s list this is. Then, we call our superclass’ #renderOwnerOn: to let the decorated component render itself. The logout link callsback the #logout method (in the actions protocol):


1 logout
2 self owner user: nil.
3 self clearAuthenticationInfo

Above, we called a couple of convenience methods, which obviously go in the convenience protocol:


1 clearAuthenticationInfo
2 login := nil.
3 password := nil.
4 passwordConfirmation := nil.
5 authenticated := false
6
7 failAuthentication: aMessage
8 authenticationMessage := aMessage.
9 password := nil.
10 passwordConfirmation := nil.
11
12 failRegistration: aMessage
13 registrationMessage := aMessage.
14 password := nil.
15 passwordConfirmation := nil.

Lastly, a couple of accessor methods:


1 login
2 ^ login
3
4 login: aLogin
5 login := aLogin
6
7 password
8 ^ password
9
10 password: aPassword
11 password := aPassword
12
13 passwordConfirmation
14 ^ passwordConfirmation
15
16 passwordConfirmation: aPassword
17 passwordConfirmation := aPassword
18
19 userDatabase
20 ^ TodoComponent userDatabase

TodoApp: our first real component

TodoApp is our root application component. It should thus register itself as a Seaside root. Let’s begin by declaring the class:


1 TodoComponent subclass: #TodoApp
2 instanceVariableNames: ‘viewers user’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-UI-Seaside’

Then, class side, in protocol testing, we implement #canBeRoot:


1 canBeRoot
2 ^ true

Still class side, in protocol initialization, add:


1 initialize
2 super initialize.
3 self registerAsApplication: #todo

This registers TodoApp as an application in Seaside, under /seaside/todo. Back on the instance side, in protocol initialization, we implement:


1 initialize
2 viewers := OrderedCollection new.
3 self addDecoration: (TodoAuthDecorator new).
4 self registerForBacktracking.

The root component knows that it needs authentication, so it immediately adds a decoration to itself. Seaside requires a component to register itself for backtracking if it’s collection of children will change during it’s lifecycle. Since the user will add and remove todos, our collection of TodoViewer instances will change.

Components render themselves, so put this in protocol rendering:


1 renderContentOn: html
2 html orderedList: [
3 self children do: [:each |
4 html listItem: [html render: each]]].
5 html paragraph: [
6 (html anchor) on: #newTodo of: self]

We render our collection of children, which is the viewers instance variable. Again, we have a callback when creating a new todo. In protocol call/answer, we add:


1 newTodo
2 | editor |
3 editor := TodoEditor new
4 todo: (Todo new);
5 yourself.
6 (self call: editor) ifNotNilDo: [:todo | self addTodo: todo]

Again, we have a couple of accessors which are pretty simple:


1 addTodo: aTodo
2 user addTodo: aTodo.
3 viewers add: (TodoViewer new todo: aTodo)
4
5 children
6 ^ viewers
7
8 user
9 ^ user
10
11 user: aUser
12 user := aUser.
13 viewers := OrderedCollection new.
14 user
15 ifNotNil: [
16 user todos do: [:each |
17 viewers add: (TodoViewer new todo: each)]]

#addTodo: adds the todo to the user instance, and also registers a new TodoViewer instance. #user: takes care of cleanup in case of logout (aUser isNil), and registers new TodoViewer instances when logging in (aUser notNil).

TodoViewer: A simple model viewer

This class is pretty simple. It’s job is to display a todo and allow it to be marked completed. Let’s declare the class:


1 TodoComponent subclass: #TodoViewer
2 instanceVariableNames: ‘todo’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-UI-Seaside’

We begin by rendering the component:


1 renderContentOn: html
2 (html paragraph)
3 class: (todo isCompleted ifTrue: [‘complete’]);
4 with: [
5 self renderDescriptionOn: html.
6 self renderCompletionStatusOn: html]

The call to #class: sets up an HTML class on the paragraph element, to help styling. A couple more rendering methods:


1 renderDescriptionOn: html
2 html html: todo description.
3 html space.
4
5 renderCompletionStatusOn: html
6 todo isCompleted
7 ifFalse: [
8 (html anchor) callback: [todo markCompleted]; with: ’It’’s done!‘]
9 ifTrue: [
10 html span: todo completedAt displayString]
11
12 style
13 ^’
14 .complete { color: #999; }
15 .complete span { font-size: smaller; }
16 }’

And the usual accessors:


1 todo
2 ^ todo
3
4 todo: anObject
5 todo := anObject

TodoEditor: Create or edit a Todo instance

The current version of the todo app doesn’t use TodoEditor to edit existing todos, but it does use it for creating new instances. As usual, let’s declare the class:


1 TodoComponent subclass: #TodoEditor
2 instanceVariableNames: ‘todo’
3 classVariableNames: ‘’
4 poolDictionaries: ’’
5 category: ‘Todo-UI-Seaside’

Next, we render the component:


1 renderContentOn: html
2 html form: [
3 html div: [
4 (html label) for: ‘description’; with: ‘Description’.
5 (html textInput) id: ‘description’; on: #description of: todo].
6 (html submitButton) on: #save of: self.
7 html space.
8 (html anchor) on: #cancel of: self]

Here is where things get interesting: #on:of: is used to set a callback on the todo instance. Seaside will call #description: of the todo instance to set the value on form post. We don’t need a temporary variable to hold the description in the component: the todo instance takes care of that.

In protocol call/answer, we add the following methods, which are called from #renderContentOn:


1 save
2 self answer: todo
3
4 cancel
5 self answer: nil

And the final accessors:


1 todo
2 ^ todo
3
4 todo: aTodo
5 todo := aTodo

Total line count: 233 lines, thanks to:


1 (Smalltalk allClasses
2 select:[:each | each category beginsWith: ‘Todo-’])
3 inject: 0 into: [:sum :each | sum + each linesOfCode]

Print this to get the total.

UPDATE 2007-10-15: I counted the lines initially by doing a fileOut and using standard command line tools: cat and wc. Seems I was slightly off.

Todos (pun intended)

There are a couple of things I’d like this todo app to be able to do:

  • Use an inline editor to change the description;
  • Use Scriptaculous to add a couple of effects;
  • Use Ajax to mark completed items;
  • Format dates and times;
  • Maybe set the user’s timezone and show dates and times using the user’s timezone;
  • Date/time formats per-user;
  • Enhance security by not storing plain-text versions of passwords in user instances.

This is a toy project. I might never again touch this application.

Recap

One point I’d like to be clear on:

Don’t do that!

I store an unencrypted copy of the password in TodoUser instances. That should never be done. As many others before me, I tried to simplify the code as much as possible. There probably lurks a Password model object in there.

Also, I am no expert on Smalltalk, Squeak or Seaside. There are probably a couple of things I could have done differently, and I hope some people out there might be interested in helping me learn more about Seaside.

I hope you enjoyed this article. There might be more of these coming later. Send me E-Mail at francois@teksol.info or write a comment.

See all 1 articles in authentication

Functional 1 articles

One thing’s for sure, it’s not obvious what’s going on when a test fails because of cookies. For that reason, I shied away from using cookies significantly.

Anyway, here’s a failure I was getting:



1 1) Failure:
2 test_create_session_with_right_username_and_password_and_remember_me(CreateSessionsControllerTest)
3 [test/functional/sessions_controller_test.rb:57:in `test_create_session_with_right_username_and_password_and_remember_me’
4 /home/francois/src/config/../vendor/plugins/mocha/lib/mocha/test_case_adapter.rb:19:in `run’]:
5 No :auth_token cookie in the response
6 -
7 auth_token:
8


1 1) Failure:
2 test_create_session_with_right_username_and_password_and_remember_me(CreateSessionsControllerTest)
3 [test/functional/sessions_controller_test.rb:57:in `test_create_session_with_right_username_and_password_and_remember_me’
4 /home/francois/src/config/../vendor/plugins/mocha/lib/mocha/test_case_adapter.rb:19:in `run’]:
5 No :auth_token cookie in the response
6 -
7 auth_token:
8 – cookie value


Turns out the @response.cookies Hash is not indifferent. You really have to use a String to get to the content:


1 def test_cookie_set
2 assert_equal ["some value"], response</span>.cookies[<span class="s"><span class="dl">&quot;</span><span class="k">auth_token</span><span class="dl">&quot;</span></span>], <span class="no">3</span> <span class="s"><span class="dl">&quot;</span><span class="k">this will succeed</span><span class="dl">&quot;</span></span> <span class="no">4</span> assert_equal [<span class="s"><span class="dl">&quot;</span><span class="k">some value</span><span class="dl">&quot;</span></span>], <span class="iv">response.cookies[:auth_token],
5 "this will always fail"
6 end

See all 1 articles in functional

Mor3 1 articles

At the last Montreal on Rails, I presented Piston. James Golick is taking care of the videos, and I promised I would put the files up. Here they are, at long last. As soon as I have a link for the video, I’ll put it up here.

The files which I used to do the presentation are OpenOffice.org 2:

Marc-André got my only hard-copy of the cheatsheet, authographed by yours truly. Way to go Marc !

The presentation went rather well. Probably due to some parsing issues, the slides didn’t look that good on Keynote, and Carl’s machine had trouble switching slides fast enough for my talk. I was surprised at the many questions from the group. Thanks for your time everyone. I really appreciated seeing you there.

See all 1 articles in mor3

Background processes 1 articles

Starting from a fresh Rails application (I’m using 2.0.2), install AttachmentFu:


1 script/plugin install http://svn.techno-weenie.net/projects/plugins/attachment_fu/

Edit config/amazon_s3.yml and put this:

config/amazon_s3.yml

1 development:
2 bucket_name: amazon-sqs-development-yourname
3 access_key_id: "your key"
4 secret_access_key: "your secret access key"
5 queue_name: amazon-sqs-development-resizer-yourname

queue_name is new. AttachmentFu does not require this, but we are going to reuse the file from our own code, so better put all configuration in the same place.

Generate a scaffolded Photo model using:


1 $ script/generate scaffold photo filename:string size:integer content_type:string width:integer height:integer parent_id:integer thumbnail:string

Edit app/views/photos/new.erb.html and replace everything with this:

app/views/photos/new.erb.html

1 <h1>New photo</h1>
2
3 <%= error_messages_for :photo >
4
5 < form_for(@photo, :html => {:multipart => true}) do |f| >
6 <p>
7 <label for="photo_uploaded_data">File:</label>
8 <= f.file_field :uploaded_data >
9 </p>
10
11 <p>
12 <= f.submit "Create" >
13 </p>
14 < end >
15
16 <= link_to ‘Back’, photos_path %>


What we did here is simply tell Rails to use a multipart encoded form, and to only provide us with a single file upload field.

Edit app/models/photo.rb and add the AttachmentFu plugin configuration:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 has_attachment :content_type => :image, :storage => :s3
3 validates_as_attachment
4 end

Start your server and confirm you can upload a file. No thumbnails were generated as we did not configure any thumbnailing to do. We don’t actually want AttachmentFu to handle that, so we can’t just specify it in the has_attachment call.

To use RightScale’s AWS SQS component, we have to configure it with the access key and secret access key. Add this to the end of the Photo class:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def queue
3 self.class.queue
4 end
5
6 class << self
7 def queue
8 # This creates the queue if it doesn’t exist
9 queue</span> ||= sqs.queue(aws_config[<span class="s"><span class="dl">&quot;</span><span class="k">queue_name</span><span class="dl">&quot;</span></span>]) <span class="no"><strong>10</strong></span> <span class="r">end</span> <span class="no">11</span> <span class="no">12</span> <span class="r">def</span> <span class="fu">sqs</span> <span class="no">13</span> <span class="iv">sqs ||= RightAws::Sqs.new(
14 aws_config["access_key_id"], aws_config["secret_access_key"],
15 :logger => logger)
16 end
17
18 def aws_config
19 return aws_config</span> <span class="r">if</span> <span class="iv">aws_config
20
21 aws_config</span> = <span class="co">YAML</span>.load(<span class="co">File</span>.read(<span class="co">File</span>.join(<span class="co">RAILS_ROOT</span>, <span class="s"><span class="dl">&quot;</span><span class="k">config</span><span class="dl">&quot;</span></span>, <span class="s"><span class="dl">&quot;</span><span class="k">amazon_s3.yml</span><span class="dl">&quot;</span></span>))) <span class="no">22</span> <span class="iv">aws_config = aws_config</span>[<span class="co">RAILS_ENV</span>] <span class="no">23</span> raise <span class="co">ArgumentError</span>, <span class="s"><span class="dl">&quot;</span><span class="k">Missing </span><span class="il"><span class="idl">#{</span><span class="co">RAILS_ENV</span><span class="idl">}</span></span><span class="k"> configuration from config/amazon_s3.yml file.</span><span class="dl">&quot;</span></span> <span class="r">if</span> <span class="iv">aws_config.nil?
24 @aws_config
25 end
26 end
27 end

#aws_config is a method that reads the configuration. #sqs is a method that provides access to an instance of RightScale::Sqs, pre-configured with the correct access keys. #queue uses #sqs to get or create a named queue. There’s also an instance version of #queue, to ease our code later on.

Let’s add the request sending:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 def send_resize_request
3 # Don’t send a resize request for thumbnails
4 return true unless self.parent_id.blank?
5
6 params = Hash.new
7 params[:id] = self.id
8 params[:sizes] = Hash.new
9 params[:sizes][:square] = "75×75"
10 params[:sizes][:thumbnail] = "100x"
11
12 begin
13 queue.push(params.to_yaml)
14 rescue
15 logger.warn {"Unable to send resize request. Error: #{$!.message}"}
16 logger.warn {$!.backtrace.join("\n")}
17
18 # Don’t raise the error so the request goes through.
19 # We don’t want the user to see a 500 error because
20 # we can’t talk to Amazon.
21 end
22 end
23 end

Now, this is getting interesting. AttachmentFu knows if the current model is a thumbnail or not by looking at parent_id. If it’s nil, we are the parent, else we are a thumbnail. We do the same thing here.

Then, we setup a couple of parameters to send to the resizer. Notice we send the actual thumbnail sizes in the message itself.

Next, we do the most important part: queue.push. This sends a message string (limited to 256 KiB) to Amazon SQS, and returns. If there is an error, we don’t actually want to prevent the request from completing, so we rescue any exceptions and log them. If you have the ExceptionNotifier plugin installed, this is a good place to log to it.

Now that we have a way to send the resize request, we have to execute it at some point. The controller is not the right place to do it. If you create Photo models from more than one controller, you’re bound to forget to call #send_resize_request. It’s better to do it in an #after_create callback, which we’ll do with a single line:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 after_create :send_resize_request
3 end

Next, we have to receive the messages. So, we write a new method in Photo:

app/models/photo.rb

1 class Photo < ActiveRecord::Base
2 class << self
3 def fetch_and_thumbnail
4 messages = queue.receive_messages(20)
5 return if messages.blank?
6
7 logger.debug {"==> Photo\#fetch_and_thumbnail — received #{messages.size} messages"}
8 messages.each do |message|
9 params = YAML.load(message.body)
10 photo = Photo.find_by_id(params[:id])
11 if photo.blank? then
12 # The Photo was deleted before we got a chance to thumbnail it.
13 # We must delete the message, or we’ll always get it afterwards.
14 message.delete
15 next
16 end
17
18 photo.generate_thumbnails(params[:sizes])
19 message.delete
20 end
21 end
22 end
23 end

The first thing we do is see if there are any messages. The call to #queue is the helper method we defined earlier on. We ask to receive up to 20 messages at a time. If there were no messages, we simply return.

Then, for each message, we have to process it, so we iterate over each message, retrieving the original parameters Hash. The important thing to do is to delete the message after we have processed it, or else the message will still be visible next time around.

#generate_thumbnails is important, but uninteresting in this discussion.

See all 1 articles in background processes

Job offer 1 articles

Pradipta's Rolodex Logo

Late last night, I had the honor of being mailed a Ruby on Rails offer. See 416 Random People with RoR on their resume + Reply All = Reverse Flash Mob for the full details.

Pretty amusing, really.

See all 1 articles in job offer

Ragan 1 articles

For the first time ever, ragan crystallized why he didn’t like the original #andand implementation, or my own "implementation"http://blog.teksol.info/2007/11/23/a-little-smalltalk-in-ruby-if_nil-and-if_not_nil, #if_not and #if_not_nil and other implementations.

His point is that if you are writing a library, and you want to use the andand gem, you, the author of the library, will pollute the Object namespace of the application that’s using our library.

In effect, the library’s author dependencies will become dependencies of the application that’s using it, even though they don’t want or need the extensions.

The same problem happens with Mongrel and Thin : all the libraries these servers use are loaded in the application that’s running on top of them.

Thank you Reginald for clearing this up for me. I now fully understand the need for the Ruby.rewrite project.

See all 1 articles in ragan

Screencast 1 articles

James Robertson of Cincom Smalltalk features my TodoApp on Seaside in 218 lines in his Smalltalk Daily series. The episode is available at smalltalk_daily-10-15-07.html

I would like to publicly thank James for taking the time to do the screencast, and assure him that the way he pronounces my last name is quite correct.

In case you aren’t following my Monticello repository, the TodoApp is now at revision 5, including a bunch of fixes Ramon Leon suggested. TodoApp also stands at 233 lines, instead of the original 218. I have a couple of refactorings in mind, so the line count will probably go down.

See all 1 articles in screencast

Passenger 1 articles

Before today, I was using Thin to host my development servers. This was a bother because I had to remember to start/stop Thins as required when I switched tasks. Well, not anymore…

Reading my feeds today, I found Passenger Preference Pane, by Fingertips.

  1. Download the preference pane itself
  2. Untar
  3. Follow the directives in the README about RubyCocoa (either Leopard 10.5.2 or install RubyCocoa yourself from source)
  4. Double click the prefpane: this will install the preference pane
  5. Open System Preferences, then Passenger in Other
  6. Drag and drop your Rails (or Rack-based) application folder into the list
  7. Click Apply
  8. Go back to System Preferences, and open Sharing
  9. Stop and start Apache using the checkbox for Web Sharing
  10. Visit http://your-apps-base-name.local/ and enjoy!

One caveat I found was that my home directory had to change permissions. I had to grant a+rx before it would work:


1 $ ls -ld ~
2 drwxr-xr-x+ 78 francois staff 2652 9 fév 11:13 /Users/francois

Of course, if you have a multi-user machine, that opens an interesting can of worms for security. A quick fix would be:


1 $ chmod a-rx * && chmod a+rx the-apps-base-name

Modify as needed.

See all 1 articles in passenger

Pledgie 1 articles

Gregory Brown, of Ruby Reports and PDF::Writer fame, wants to quit his job, sort of.

Here’s a crazy idea I just had, and I’m wondering what folks think about it.

People do open source for a lot of reasons, ranging from pragmatic to idealistic. …

What if I could just do open source for a while, non-commercially?

How much would it cost for me to do at least 80 hours a month of development on software projects such as PDF::Writer, Ruport, and some other projects I wish I had the time to get my hands on?

I did the math, and the number came out low (subjective). I could meet all my expenses and save some money for about $2000 a month. Basically, if 200 people donated $60 right now, I could take 6 months off and do nearly 500 hours of work, and that’s only if I didn’t find myself obsessed with and doing extra hours on a project.

Gregory Brown, in I’d love to quit my job!

Gregory started a Wiki where he documented his proposal. Head on over to the Ruby Mendicant for the details.

Finally, if you’d like to donate (like I did !), head on over to pledgie:

Click here to lend your support to: Ruby Mendicant and make a donation at www.pledgie.com !

See all 1 articles in pledgie

Unit-tests 1 articles

Thoughtbot’s Shoulda is a very nice piece of work. Their ActiveRecord macros are also a god-send:


1 should_protect_attributes :family_id, :debit_account_id, :credit_account_id, :created_at, :updated_at

This code will assert that ActiveRecord attributes are somehow protected (either though attr_accessible or attr_protected). But what about the reverse? There isn’t a macro to do that. I happened to need it, so I implemented it on my own fork of Shoulda.

This allows us to specify:


1 should_protect_attributes :family_id, :debit_account_id, :credit_account_id, :created_at, :updated_at
2 should_allow_attributes :family, :debit_account, :credit_account, :amount, :posted_on, :description

NOTE: Example taken from my family budget application.

See all 1 articles in unit-tests

Links 1 articles

James Golick just opened our company blog by adding FLOSS week. Over the next few days we’ll release a couple of plugins that we’ve been working on/with for the past few projects.

Only problem is, we never get around to releasing this stuff. Starting a blog seemed like a good excuse to spend some time polishing up some code and, you know, writing READMEs.

Go on over and enjoy the view! Don’t forget the RSS feed: GiraffeSoft: The Blog

See all 1 articles in links

Tech-support 1 articles

I’m a happy DreamHost customer, for many years. Recently, they moved my data to a new server. That in itself wasn’t a problem, but this blog went down with their “bad_httpd_conf” error. DreamHost support has always been prompt to answer my questions. But more importantly, when a customer asks for support, they have answer right where it matters.

As part of the support process, DreamHost asks what site you have a problem with. Then they run some diagnostics on the network and report those as warnings.

DreamHost Support gives you diagnostics when you ask for support on a site. In this case, DreamHost is telling me my DNS servers are non-DreamHost, which might cause problems.
Click to view full size

After they’ve done that, DreamHost asks for what type of problem you have. And the different categories have tips you can apply to try to correct the problem, before you have to wait for DreamHost support.

DreamHost Support offers tips on fixing your problems before waiting for them, in this case telling me that editing and saving the site might correct the "bad_httpd_conf" error
Click to view full size

Another thing that’s very nice: after logging in, one of the 8 main options is, you guessed it, Support. DreamHost doesn’t hide their support. It’s very visible. I’m very satisfied with the services I use.

See all 1 articles in tech-support

Puppet 1 articles

I do more and more Puppet, and I really like it. It’s a simple way to declare the desired state of the world. As I do more and more with Puppet, I needed to debug my manifests: check that they were doing the expected thing. I was looking for some kind of dry run option, similar to apt-get‘s or other tools, but puppet help apply didn’t provide any helpful hints. Until I stumbled upon Puppet dry run.

A direct run with --noop --test failed because --test was not recognized, but --noop was!. A sample run with --noop looks like this:


1 # puppet apply puppet/hosts/db03.staging.internal.pp —noop
2 notice: /Stage[main]//Postgresql::Database[svreporting_staging]/Exec[createdb svreporting_staging]/returns: current_value notrun, should be 0 (noop)
3 notice: Postgresql::Database[svreporting_staging]: Would have triggered refresh from 1 events
4 notice: /Stage[main]//Cron[replace svreporting_staging with svanalytics_staging]/ensure: current_value absent, should be present (noop)
5 notice: Class[Main]: Would have triggered refresh from 2 events
6 notice: Stage[main]: Would have triggered refresh from 1 events
7 notice: Finished catalog run in 10.86 seconds

Check the wording: Would have triggered, and should be. These are great ways to know what Puppet will do for you. There is only one caveat: when you need to spend more than a few minutes debugging your recipes, disable the automatic run. The scheduled run from puppet agent may go behind your back and apply your recipe, which you’re just debugging. In my case, this is very easy to do since I host my puppet manifests in a Git repository, which is pulled hourly and applied.

I’m only waiting for my pull request to be applied so you don’t have to go through the same process as I did.

See all 1 articles in puppet

Csv 1 articles

The way I’m doing it here is obsolete, and Returning CSV data to the browser – updated has a better version.

Over on the Rails mailing list, Pete asks:

One of my apps has to export data for the backend system to process it. What’s the best way to create a CSV file in Rails and then return it as
a file to the client?

I have used two tricks in the past to export the data.

Exporting using CSV::Writer

This is simple. Use CSV::Writer and ActionController’s send_data method.

app/controllers/report_controller.rb

1 class ReportController < ApplicationController
2 def report
3 models</span> = <span class="co">Model</span>.find(<span class="sy">:all</span>, <span class="sy">:conditions</span> =&gt; [<span class="s"><span class="dl">'</span><span class="k">...</span><span class="dl">'</span></span>]) <span class="no"> 4</span> report = <span class="co">StringIO</span>.new <span class="no"> <strong>5</strong></span> <span class="co">CSV</span>::<span class="co">Writer</span>.generate(report, <span class="s"><span class="dl">'</span><span class="k">,</span><span class="dl">'</span></span>) <span class="r">do</span> |csv| <span class="no"> 6</span> csv &lt;&lt; <span class="s"><span class="dl">%w(</span><span class="k">Title Total</span><span class="dl">)</span></span> <span class="no"> 7</span> <span class="iv">models.each do |model|
8 csv << [model.title, model.total]
9 end
10 end
11
12 report.rewind
13 send_data(report.read,
14 :type => text/csv; charset=iso-8859-1; header=present,
15 :filename => report.csv)
16 end
17 end

By default, send_data will make the Content-Disposition header equal to attachment. This will ask the browser to download the file, instead of displaying it in the browser window.

Exporting using a regular view

The second method is even simpler:

app/controllers/report_controller.rb

1 class ReportController < ApplicationController
2 def report
3 @models = Model.find(:all, :conditions => [])
4 response.headers[Content-Type] = text/csv; charset=iso-8859-1; header=present
5 response.headers[Content-Disposition] = attachment; filename=report.csv
6 end
7 end

app/views/report/report.rhtml

1 Title,Value
2 <% @models.each do |model| -%>
3 "<%= model.title.gsub(‘"’, ‘""’) >",<= model.value >
4 < end -%>


Notice how I escape quotes in the view ? This is important, or else your parsing will be broken when you read the file back in. Of course, if you use CSV::Writer, you won’t have to muck with that – it will all be taken care of for you.

UPDATE 2006-03-30: Changed Content-Type from text/comma-separated-values to text/csv, and added the header optional parameter, per RFC 4180

UPDATE 2007-04-26: Just noticed that the report method above would render with a layout if one were defined in ApplicationController. Simply call #render and tell it to use no layout: render :action =&gt; :report, :layout =&gt; false

See all 1 articles in csv

Debugging 1 articles

I was meta-meta-programming, and had an issue when an #included method was called:


1 anon = Module.new
2 anon.class_eval do
3 def self.included(base)
4 debugger
5 if some_method then
6 # code
7 else
8 # more code
9 end
10 end
11 end

The code above resulted in:


1 /Users/francois/Projects/project/lib/extensions.rb:214:in `included’: undefined local variable or method `some_method for #<Module:0×103305b20> (NameError)
2 from /Users/francois/Projects/project/lib/extensions.rb:245:in `include

3 from /Users/francois/Projects/project/lib/extensions.rb:245:in `send’
4
5

The debugger statement above just wouldn’t take: Ruby ran right over it. I happened to look at ruby-debug’s Rubygem spec file:


1 - !ruby/object:Gem::Specification
2 name: ruby-debug
3
4 executables:
5 – rdebug
6

Oh, had never noticed the executables before… Sure enough, I managed to run under debugger control immediately:


1 $ rdebug -I test test/functional/ad_spots_controller_test.rb [-4, 5] in /Users/francois/Projects/bloom/adgear-admin/test/functional/ad_spots_controller_test.rb
2 => 1 require "test_helper"
3 2
4 3 class AdSpotsControllerTest < ActionController::TestCase
5 4
6 5 def setup
7 /Users/francois/Projects/bloom/adgear-admin/test/functional/ad_spots_controller_test.rb:1
8 require "test_helper"
9 (rdb:1)

From there, I was able to use (C)ontinue to end up on my debugger statement. Happy times ensued!

For the curious, the NoMethodError is because #some_method is defined on base, not on self. self in this context is the included module, while base is the place where we’re including it into.

See all 1 articles in debugging

Platform 1 articles

I just stumbled upon Is Windows a First Class Platform for Ruby? by Peter Cooper and Is Windows a supported platform for Ruby? I guess not by Luis Lavena.

I have to admit, I did use Windows daily for 3 years before I switched permanently to Ubuntu, about 6 months back. Initially, I was using IntelliJ’s IDEA as my development platform, then I switched to the e Text Editor. I never had many problems with gems that didn’t/couldn’t/wouldn’t install. There are many kind souls in the community that keep these gems up to date. I’m talking specifically about Tim Hunter (RMagick), Luis Lavena (Mongrel) and others.

Just before I made the final switch, I was using Linux in a VMware image to run XLsuite. The application was unbearably slow in Windows, but acceptable in a virtual machine. I was using the e Text Editor as my editor, accessing the code through a Samba share.

My own experience was pretty positive. Now that I’m on Ubuntu, I wouldn’t go back though. What made me switch ? Better performance on the same hardware, mostly; the novelty of the experience. I do not dislike Windows, nor do I think Redmond is a bad place. Windows is a fine platform.

Just to contrast, look at the excellent support Java enjoys on Windows. Windows is the 2nd platform for Java (with Solaris being the 1st). We, the Ruby community, should be learning from Sun. There are many, many more Windows machines than Mac or Linux machines out there. There are literally millions of people who could learn to write Ruby, but are on Windows. Dr Nic said it all: … People Use Windows Too. Whether you want to or not, Windows isn’t going away soon.

Personally, I have made the switch. But just on my small team, Windows users outnumber other platforms 1 to 1. Here’s the breakdown:

  • 4 Windows (2 coders, 1 designer, 1 sponsor)
  • 2 Ubuntu (1 coder, 1 designer)
  • 1 Mac (1 ExtJS coder)

Is Windows a good platform for Ruby ? Yes. Is Windows a great platform for Ruby ? No, but there’s no reason why it shouldn’t be. Just look at Why‘s work on Hackety Hack. I’m really impressed and happy that I will be able to show Ruby to my daughters. The catch ? Hackety Hack is for Windows only right now.

Please, let’s keep and increase Windows support. Once they’re hooked, they might switch, who knows ?

See all 1 articles in platform

Callbacks 1 articles

In one of my projects, I wanted to prevent instances of particular classes to be deleted if they were in any way associated to another object. In database terms, I wanted an ON DELETE RESTRICT constraint.

Since I cannot rely on the database to enforce it for me (MySQL 4, MyISAM engine), I coded the following:

test/unit/city_test.rb

1 require File.dirname(FILE) + /../test_helper
2
3 class CityTest < Test::Unit::TestCase
4 fixtures :cities, :contacts
5
6 def setup
7 city</span> = <span class="co">City</span>.find(<span class="sy">:first</span>) <span class="no"> 8</span> <span class="r">end</span> <span class="no"> 9</span> <span class="no"><strong>10</strong></span> <span class="r">def</span> <span class="fu">test_prevent_destruction_if_associated_to_any_contact</span> <span class="no">11</span> <span class="iv">city.contacts << contacts(:jill)
12 city</span>.destroy <span class="no">13</span> assert_not_nil <span class="co">City</span>.find(<span class="iv">city.id), should not have been destroyed
14 assert_match /cannot destroy.*contacts/i, city</span>.errors.on_base, <span class="no"><strong>15</strong></span> <span class="s"><span class="dl">'</span><span class="k">reports error condition to user</span><span class="dl">'</span></span> <span class="no">16</span> <span class="r">end</span> <span class="no">17</span> <span class="no">18</span> <span class="r">def</span> <span class="fu">test_allow_destruction_if_not_associated_to_any_contact</span> <span class="no">19</span> <span class="iv">city.destroy
20 assert_raises ActiveRecord::RecordNotFound do
21 City.find(@city.id)
22 end
23 end
24 end

app/models/city.rb

1 class City < ActiveRecord::Base
2 has_and_belongs_to_many :contacts, :join_table => contacts_cities
3
4 def destroy
5 unless self.contacts.empty?
6 self.errors.add_to_base \
7 We cannot destroy this instance since one or more contacts refer to it)
8 return
9 end
10
11 super
12 end
13 end

I had to override destroy because in has_and_belongs_to_many relationships, Rails deletes join table records before deleting the record. This means that in before_destroy filters, self.contacts.empty? will always report true. Ticket #1183: dependents are destroyed before client before_destroy hooks are called is already opened on this issue.

UPDATE (2006-03-08) Fixed since 2006-02-13

See all 1 articles in callbacks

Open source 1 articles

I just received 2 contributions from Josh Nichols:

  • New —repository-type option on piston import to force the repository backend to use (instead of letting Piston guess), and for cases where Piston is unable to guess: ea958dd;
  • Test suite reorganization: 1cef7b6 and 9cfa8f3

Both contributions were accepted and are now part of Piston’s master branch. Thank you very much, Josh, for your work.

If you want to help, do not fear !


1 $ git clone git://github.com/francois/piston.git
2 $ # make changes
3 $ git commit
4 $ # fork piston’s repository
5 $ git remote add github git@github.com:YOURNAME/piston.git
6 $ git push github master
7 $ # Send me a pull request

See all 1 articles in open source

Activeresource 1 articles

I have a work-in-progress branch for using Digest authentication with ActiveResource.

So, how did I do it? It wasn’t too hard actually. When I spiked, I changed ActiveResource::Connection#request to handle authentication itself. I ended up with a big mess: a new rescue clause, 10 lines of code to calculate the digest and so on. But I knew it would work. So, I git checkout . and started with tests, as it should.

The way ActiveResource is built, if a username / password is sent in, ActiveResource will send those automatically in an Authorization header, using the Basic authentication method. I need a way to turn this off. Thus grew #use_basic_authentication= and #use_digest_authentication=.

Next up, actually being able to calculate the Digest. A quick search turned up code by Eric Hodel in the form of a Ruby module: An implementation of HTTP Digest Authentication in Ruby

After a bit of cleanup and rewriting, I have a branch of ActiveResource that’s ready to be commented on. Please see francois/ar_digest and leave comments there.

An example of using Digest would be:


1 require "logger"
2 require "activeresource"
3 require "pp"
4
5 ActiveResource::Base.logger = Logger.new(STDERR)
6
7 ActiveResource::Base.site = "http://adgear.local/api"
8 ActiveResource::Base.user = "francois"
9 ActiveResource::Base.password = "my-funny-new-password-which-you’ve-never-seen-before"
10 ActiveResource::Base.timeout = 30
11
12 # Don’t attempt Basic authentication, but be sure to use Digest
13 ActiveResource::Base.use_basic_authentication = false
14 ActiveResource::Base.use_digest_authentication = true
15
16 class Site < ActiveResource::Base
17 end
18
19 pp sites = Site.find(:all)

This work was sponsored by Bloom Digital Platforms, as part of my work on their AdGear API.

See all 1 articles in activeresource

Html email 1 articles

I needed to send E-Mail as part of my work on XLsuite, and I had problems with HTML E-Mail. I know, I know, it’s preferable to have plain text and all, but user requirements being what they are…

So, anyway, long story short, it didn’t work. I just can’t use the multiple views with content type embedded in them, as the HTML’s text is written from the user interface. So, I turned to the trusted Google, and found an older article on court3nay website How to send multipart/alternative e-mail with inline attachments. The essence of the trick is to use a multipart/related part, and embed the HTML content inside this part. Enjoy !

See all 1 articles in html email

Actionmailer 1 articles

I needed to send E-Mail as part of my work on XLsuite, and I had problems with HTML E-Mail. I know, I know, it’s preferable to have plain text and all, but user requirements being what they are…

So, anyway, long story short, it didn’t work. I just can’t use the multiple views with content type embedded in them, as the HTML’s text is written from the user interface. So, I turned to the trusted Google, and found an older article on court3nay website How to send multipart/alternative e-mail with inline attachments. The essence of the trick is to use a multipart/related part, and embed the HTML content inside this part. Enjoy !

See all 1 articles in actionmailer

Ar 1 articles

Does anyone know why ActiveRecord does not merge joins when using scopes ?

Scopes are wonderful, thanks for has_finder and named scopes, but joins aren’t supported. This code:


1 City.nearest(latitude, longitude).find_tagged_with(tags)

raises an ActiveRecord::StatementInvalid complaining that a specified column could not be found. The column can’t be found because the join was dropped on the floor, instead of being merged.

The change me and Marc-André are proposing to make is this:


1 $ git diff 36236235039b3b0cab22a24ce7b45a8ec071cb5e 45e7c94be02a586f70908694545e8484e4a85382 vendor
2 diff -git a/vendor/rails/activerecord/lib/active_record/base.rb b/vendor/rails/activerecord/lib/active_record/base.rb
3 index 261d854..427e3ef 100755
4 a/vendor/rails/activerecord/lib/active_record/base.rb
5 + b/vendor/rails/activerecord/lib/active_record/base.rb
6 @</span> -1673,6 +1673,8 <span class="chg">@ module ActiveRecord #:nodoc:
7 hash[method][key] = [params[key], hash[method][key]].collect{ |sql| "( %s )" % sanitize_sql(sql) }.join(" AND "
8 elsif key == :include && merge
9 hash[method][key] = merge_includes(hash[method][key], params[key]).uniq
10 elsif key == :joins && merge
11 hash[method][key] = [params[key], hash[method][key]].join(" ")
12 else
13 hash[method][key] = hash[method][key] || params[key]
14 end

This is based on Rails 2.0.2, but Rails Edge has the same problem. So, any ideas ?

See all 1 articles in ar

Recruitment 1 articles

Pradipta's Rolodex Logo

Late last night, I had the honor of being mailed a Ruby on Rails offer. See 416 Random People with RoR on their resume + Reply All = Reverse Flash Mob for the full details.

Pretty amusing, really.

See all 1 articles in recruitment

Alter-ego 1 articles

I stumbled on AlterEgo last week, after Avdi announced it. This library implements the state pattern for any object, not just ActiveRecord models, which acts_as_state_machine (and it’s successor, aasm) do.

But, many people, myself included, want to use AlterEgo in the context of ActiveRecord models. Fortunately, AlterEgo provides us with all the necessary plumbing to do that very easily.

I will reuse Avdi’s traffic light example from the specifications. Avdi already points us to 90% of the solution in his example:


1 class TrafficLightWithCustomStorage
2 def state
3 gyr = [
4 hardware_controller</span>.green, <span class="no"> <strong>5</strong></span> <span class="iv">hardware_controller.yellow,
6 hardware_controller</span>.red <span class="no"> 7</span> ] <span class="no"> 8</span> <span class="no"> 9</span> <span class="r">case</span> gyr <span class="no"><strong>10</strong></span> <span class="r">when</span> [<span class="pc">true</span>, <span class="pc">false</span>, <span class="pc">false</span>] <span class="r">then</span> <span class="sy">:proceed</span> <span class="no">11</span> <span class="r">when</span> [<span class="pc">false</span>, <span class="pc">true</span>, <span class="pc">false</span>] <span class="r">then</span> <span class="sy">:caution</span> <span class="no">12</span> <span class="r">when</span> [<span class="pc">false</span>, <span class="pc">false</span>, <span class="pc">true</span>] <span class="r">then</span> <span class="sy">:stop</span> <span class="no">13</span> <span class="r">else</span> raise <span class="s"><span class="dl">&quot;</span><span class="k">Invalid state!</span><span class="dl">&quot;</span></span> <span class="no">14</span> <span class="r">end</span> <span class="no"><strong>15</strong></span> <span class="r">end</span> <span class="no">16</span> <span class="no">17</span> <span class="r">def</span> <span class="fu">state=</span>(value) <span class="no">18</span> gyr = <span class="r">case</span> value <span class="no">19</span> <span class="r">when</span> <span class="sy">:proceed</span> <span class="r">then</span> [<span class="pc">true</span>, <span class="pc">false</span>, <span class="pc">false</span>] <span class="no"><strong>20</strong></span> <span class="r">when</span> <span class="sy">:caution</span> <span class="r">then</span> [<span class="pc">false</span>, <span class="pc">true</span>, <span class="pc">false</span>] <span class="no">21</span> <span class="r">when</span> <span class="sy">:stop</span> <span class="r">then</span> [<span class="pc">false</span>, <span class="pc">false</span>, <span class="pc">true</span>] <span class="no">22</span> <span class="r">end</span> <span class="no">23</span> <span class="iv">hardware_controller.green = gyr[0]
24 hardware_controller</span>.yellow = gyr[<span class="i">1</span>] <span class="no"><strong>25</strong></span> <span class="iv">hardware_controller.red = gyr[2]
26 end
27 end

When an object implements #state and #state=, AlterEgo will serialize it’s state using these two methods. Let’s hook this to ActiveRecord (note, I changed the code for a quick example, so it’s not exactly the same, but very similar):

app/models/traffic_light.rb

1 class TrafficLight < ActiveRecord::Base
2 def state
3 case read_attribute(:color)
4 when "green"; :proceed
5 when "yellow"; :caution
6 when "red"; :stop
7 else; raise "Invalid color: #{read_attribute(:color).inspect}"
8 end
9 end
10
11 def state=(value)
12 color_value = case value
13 when :proceed; "green"
14 when :caution; "yellow"
15 when :stop; "red"
16 else; raise "Don’t know how to convert #{value.inspect} to color value"
17 end
18 write_attribute(:color, color_value)
19 end
20 end

The most important thing you should check is that your database model has a valid state on model instantiation. Whether you do it using the :default key in your migration or some other way is irrelevant, but the value has to be provided, or the #state method will barf (or implement default processing there?) In the example app, I opted to use a default value:

db/migrate/20081209150159_create_traffic_lights.rb

1 class CreateTrafficLights < ActiveRecord::Migration
2 def self.up
3 create_table :traffic_lights do |t|
4 t.string :color, :default => "red"
5 end
6 end
7
8 def self.down
9 drop_table :traffic_lights
10 end
11 end

You may check the full code in action in the GitHub repository: alter_ego_plus_active_record

See all 1 articles in alter-ego

Observer 1 articles

Active Record has a nice feature that enables the programmer to separate concerns. In the application I am building, I need to reject some transactions based on business rules. For example, if an attacker attemps to empty an account by requesting multiple payouts, or attempting to load more than a specific amount of money in the user’s account.

In this application, I am using STI (Single Table Inheritance) to represent the different types of transaction. The hierarchy looks like this:


1 class GreenbackTransaction < ActiveRecord::Base
2 end
3
4 class AccountRechargeTransaction < GreenbackTransaction
5 end
6
7 class AccountPayoutTransaction < GreenbackTransaction
8 end

I defined my observer like this:

app/models/greenback_transaction_observer.rb

1 class GreenbackTransactionObserver < ActiveRecord::Observer
2 def after_create(txn)
3 # Code to reject the transaction if it is suspicious
4 end
5 end

Using the console, everything was working fine. So, off I went to write a test for it (I know, it should have been the other way around, but I haven’t used observers much, and I wanted to see what was going on).

My test is defined like this:

test/unit/greenback_transaction_observer_test.rb

1 class GreenbackTransactionObserverTest < Test::Unit::TestCase
2 fixtures :greenback_transactions, :users, :accounts, :affected_accounts
3
4 def setup
5 sam</span> = users(<span class="sy">:sam</span>) <span class="no"> 6</span> <span class="co">Setting</span>.recharge_amount_threshold = <span class="i">150</span>.to_money <span class="no"> 7</span> <span class="r">end</span> <span class="no"> 8</span> <span class="no"> 9</span> <span class="r">def</span> <span class="fu">test_suspicious_transaction_rejected</span> <span class="no"><strong>10</strong></span> assert_nothing_raised <span class="r">do</span> <span class="no">11</span> <span class="co">AccountRechargeTransaction</span>.new(<span class="sy">:account</span> =&gt; <span class="iv">sam.account, :amount => 100.to_money)
12 end
13
14 assert_raise(TransactionFailureException) do
15 AccountRechargeTransaction.new(:account => @sam.account, :amount => 100.to_money)
16 end
17 end
18 end

To my complete surprise, this didn’t work. After much investigation, I found that the observer was not loaded for GreenbackTransaction. After some fooling around, adding logging statements in Rails core, I finally stumbled upon the solution:

app/models/account_recharge_transaction_observer.rb

1 class AccountRechargeTransactionObserver < ActiveRecord::Observer
2 observe AccountRechargeTransaction
3
4 def after_create(txn)
5 # Code to reject the transaction if it is suspicious
6 end
7 end

The problem was the observer was registered on GreenbackTransaction, and it seems the observers aren’t inherited in subclasses. This is important and bears repeating: if you use STI and observers, observe your subclasses !.

See all 1 articles in observer

Code-generation 1 articles

Not sure if this falls into the useful or into the weird category, but I’m beginning to do code generation in my behavior specifications…

Take a look at the behavior specification:

text/unit/transaction_state_test.rb

1 require File.dirname(FILE) + /../test_helper
2
3 class TransactionStateOrderingTest < Test::Unit::TestCase
4 TransactionState::StatusNames.each do |status|
5 define_method("test_#{status}query_method") do
6 assert TransactionState.send(status).send("#{status}?"), status
7 (TransactionState::StatusNames – [status]).each do |inner_status|
8 assert !TransactionState.send(status).send("#{inner_status}?"), inner
status
9 end
10 end
11 end
12 end

And the corresponding implementation:

app/models/transaction_state.rb

1 class TransactionState
2 StatusNames = %w(started approved processed completed cancelled).freeze
3
4 attr_reader :name, :index
5 alias_method :to_s, :name
6
7 def initialize(name)
8 name</span>, <span class="iv">index = name.to_s, StatusNames.index(name.to_s)
9 raise "Unknown status name: #{name.inspect}" unless index</span> <span class="no"><strong>10</strong></span> <span class="r">end</span> <span class="no">11</span> <span class="no">12</span> <span class="c"># Generate instance methods to query the state of this instance.</span> <span class="no">13</span> <span class="c"># Generates #started? and #completed?, among others.</span> <span class="no">14</span> <span class="co">StatusNames</span>.each <span class="r">do</span> |status| <span class="no"><strong>15</strong></span> define_method(<span class="s"><span class="dl">&quot;</span><span class="il"><span class="idl">#{</span>status<span class="idl">}</span></span><span class="k">?</span><span class="dl">&quot;</span></span>) <span class="r">do</span> <span class="no">16</span> <span class="pc">self</span>.name == <span class="co">TransactionState</span>.send(status).name <span class="no">17</span> <span class="r">end</span> <span class="no">18</span> <span class="r">end</span> <span class="no">19</span> <span class="no"><strong>20</strong></span> <span class="c"># Generate class methods to return pre-instantiated</span> <span class="no">21</span> <span class="c"># TransactionState objects, one per status name.</span> <span class="no">22</span> <span class="r">class</span> &lt;&lt; <span class="cl">self</span> <span class="no">23</span> <span class="cv">states_cache</span> = <span class="co">Hash</span>.new <span class="no">24</span> <span class="co">StatusNames</span>.each <span class="r">do</span> |status| <span class="no"><strong>25</strong></span> <span class="cv">states_cache</span>[status.to_sym] = <span class="co">TransactionState</span>.new(status) <span class="no">26</span> define_method(status) <span class="r">do</span> <span class="no">27</span> <span class="cv">@states_cache[status.to_sym]
28 end
29 end
30 end
31
32 Statuses = StatusNames.map {|status| TransactionState.send(status)}
33 end

Nothing too earth shattering, except for the specs. How do I ensure the specs were properly generated ? I didn’t, except for confirming that the number of tests and assertions went up as expected.

See all 1 articles in code-generation

Virtualization 1 articles

The tools you use

2006-12-15

Signal vs Noise asked today to tell them The tools you use. Since I have a new environment, I thought I’d share it here instead.

I am on Windows. I know, not the best. But, with a little help from virtualization technology, it’s not as bad as it sounds. I’ll get the easy bits out:

A few days ago, I wanted to try Ubuntu, just for kicks. I downloaded Ubuntu Server, and installed it on VMware Server, a free virtualization product for Windows. After setting up, I wanted something to play with, so I copied a Rails application I’m working on at the moment. I ran my tests, and I couldn’t believe my eyes. My tests were running faster than if they were run on Windows.

So, I took out a wristwatch, and ran the tests on both sides. Yup, Virtual Linux was faster than Windows. By a good margin too: 25 seconds vs 7. That’s from hitting Enter after “rake test:units” until the command line is back again. Obviously, YMMV.

So, I installed Samba on Linux, setup a shared drive, and am now happily editing using the e Text Editor, while running my tests under Linux, and serve the application using Mongrel, on Linux.

So, my development environment is:

If you are on Windows, I really encourage you to try this. It was worth it for me.

See all 1 articles in virtualization

Vim 1 articles

I wanted to paste some HTML into Vim, and I was having a hard time getting something that preserved existing indentation.

After doing :help autoindent, I found out about smartindent. So, I turned both smartindent and autoindent off before doing my paste. That worked wonders!


1 :set nosmartindent
2 :set noautoindent
3 i
4 <CMD>-V
5 <ESC>
6 :set smartindent
7 :set autoindent

Voilà, a correctly pasted, non-indented version of whatever was in the clipboard!

UPDATE: 2 minutes after I posted the link to this article on Twitter, @jpalardy replied:

See all 1 articles in vim

Xunit 1 articles

If you are exploring Erlang and want an xUnit implementation, look no further than eunit.

UPDATE 2007-04-16: I did not build eunit. Richard Carlsson (the current maintainer) pointed me to it on the erlang-questions mailing list.

I started writing my own, and I will document that in the coming days as it was a very good learning experience for me.

Erlang ? Yes, I like it. A lot. I bought Programming Erlang as a beta book from the Pragmatic Programmers to help me get started faster.

My plan is to build Battleship in Erlang. That should prove very interesting.

See all 1 articles in xunit

Mysql 1 articles

Ouch! That hurts! Thurdsay (yesterday) I finished setting up new DB instances for XLsuite. All was well, and the servers performed admirably well for 24 hours. Then, suddenly, the LVM partition that held /var/lib/mysql went away…

Around 18:08 UTC (11:08:09 PDT), our main MySQL database server went down. Luckily, yesterday (Thurdsay), I had just replaced our whole DB infrastructure to have a replicated master/slave setup. It took us 15 minutes to notice that the sites were down, and another 20 minutes to execute a database failover. By 18:50 UTC (11:50 PDT), things were back to normal.

François Beausoleil

Unfortunately, I did not have the scripts in place yet to do the database failover, so it was a manual process. It wasn’t too hard, but I did have to remember one thing:


1 Mysql::Error: The MySQL server is running with the —read-only option so it cannot execute this statement: INSERT INTO sessions (`updated_at`, `sessid`, `data`) VALUES

In High Performance MySQL, the authors recommend, in Chapter 8:

On the slave, we recommend enabling the following configuration options:

skip_slave_start
read_only

Double oops! Anyway, the advice in the book was invaluable to me. If you manage MySQL and don’t already have the book, I highly suggest you buy it.

See all 1 articles in mysql

Strange 1 articles

I really don’t know how I did this. I’d like to say it’s the Gremlins in the machine, but it must be some kind of human error.


1 ~ $ gem -version
2 1.2.0
3 ~ $ gem env
4 RubyGems Environment:
5 – RUBYGEMS VERSION: 1.2.0
6 – RUBY VERSION: 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
7 – INSTALLATION DIRECTORY: /Library/Ruby/Gems/1.8
8 – RUBY EXECUTABLE: /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
9 – EXECUTABLE DIRECTORY: /usr/bin
10 – RUBYGEMS PLATFORMS:
11 – ruby
12 – universal-darwin-9
13 – GEM PATHS:
14 – /Library/Ruby/Gems/1.8
15 – /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8
16 – GEM CONFIGURATION:
17 – :update_sources => true
18 – :verbose => true
19 – :benchmark => false
20 – :backtrace => false
21 – :bulk_threshold => 1000
22 – "install" => "
-env-shebang"
23 – "update" => "—env-shebang"
24 – :sources => ["http://gems.rubyforge.org", "http://gems.github.com/"]
25 – REMOTE SOURCES:
26 – http://gems.rubyforge.org
27 – http://gems.github.com/
28 ~ $ sudo gem update —system
29 Password:
30 Updating RubyGems
31 Nothing to update
32 ~ $ rails newapp
33 create
34 create app/controllers
35
36 create log/test.log
37 ~ $ cd newapp/
38 ~/newapp $ script/generate model person
39 Rails requires RubyGems >= 1.3.1 (you have 1.2.0). Please `gem update —system` and try again.
40
41 $ ls -d /Library/Ruby/Gems/1.8/gems/rubygems*
42 /Library/Ruby/Gems/1.8/gems/rubygems-update-1.2.0 /Library/Ruby/Gems/1.8/gems/rubygems-update-1.3.1
43 /Library/Ruby/Gems/1.8/gems/rubygems-update-1.3.0

What’s going on? Anybody has an idea? A quick Google search didn’t turn up anything interesting. Probably I’m not searching for the right thing.

See all 1 articles in strange

Logging 1 articles

Today, I was replacing RailsCron and BackgrounDRb with generated Daemons, and I happened to investigate how Rails sets up it’s logger. For the longest time, I knew it was possible to change the options, but I just never investigated how to do it. I thought I’d share my findings so others won’t be in the dark as I was.

Actually doing the replacement is very easy:

config/environment.rb

1 Rails::Initializer.run do |config|
2 config.logger = Logger.new(File.dirname(FILE) + "/../log/#{RAILS_ENV}.log")
3 end

That’s it. Have fun. Stop reading… Unless you want more details.

This is essentially what Rails does, except now you have complete control over how the Logger is instantiated.

Some options you might want to investigate:


1 # Keep at most 2 2 megabytes log files
2 config.logger = Logger.new(File.dirname(FILE) + "/../log/#{RAILS_ENV}.log", 2, 210241024)
3
4 # Create a new log file each day
5 config.logger = Logger.new(File.dirname(FILE) + "/../log/#{RAILS_ENV}.log", "daily")

If you are running a script which loads the Rails environment manually, you can also do this:

lib/daemons/futures_worker.rb

1 #!/usr/bin/env ruby
2
3 # You might want to change this
4 raise "No RAILS_ENV defined" if ENV["RAILS_ENV"].to_s.empty?
5
6 require "logger"
7
8 RAILS_DEFAULT_LOGGER = Logger.new(File.dirname(FILE) + "/../../log/futures_runner.rb.log", 3, 210241024)
9 require File.dirname(FILE) + "/../../config/environment"
10
11 # Other code as appropriate

The Rails::Initializer is smart enough to use either the RAILS_DEFAULT_LOGGER or the one defined in the configuration block. For the gory details, please read Rails::Initializer#initialize_logger

See all 1 articles in logging

Tip 1 articles

Rails deprecations are a nice thing. They allow us to know what transition path to use, instead of being left in the dark. With deprecated functionality being removed from Rails, it’s important to know these things.

If you have a decent test coverage, you might want to add this to your config/environments/test.rb:

config/environments/test.rb

1 class DeprecatedFunctionality < SyntaxError; end
2 ActiveSupport::Deprecation.behavior = Proc.new {|message, callstack|
3 raise DeprecatedFunctionality, message
4 }

This will raise an exception instead of simply reporting the warning to the console. If you have CruiseControl.rb installed, the messages will show up as errors there too. This is a quick and easy way to force yourself to actually use the new functionality.

See all 1 articles in tip

Email 1 articles

I needed to send E-Mail as part of my work on XLsuite, and I had problems with HTML E-Mail. I know, I know, it’s preferable to have plain text and all, but user requirements being what they are…

So, anyway, long story short, it didn’t work. I just can’t use the multiple views with content type embedded in them, as the HTML’s text is written from the user interface. So, I turned to the trusted Google, and found an older article on court3nay website How to send multipart/alternative e-mail with inline attachments. The essence of the trick is to use a multipart/related part, and embed the HTML content inside this part. Enjoy !

See all 1 articles in email

Migration 1 articles

However you cut it, sometimes there are differences between the development and production environments.

For example, if you are integrating your website with PayPal, you probably want your local tests in the development environment to hit https://developer.paypal.com/, and not the main site at https://www.paypal.com/.

In my applications, I always store the URL to the PayPal service in a Setting or Configuration object. This allows me the flexibility of changing the value whenever I need to.

Of course, I use Migrations to generate my tables, or pre-populate them with data. Here is a typical migration:

db/migrate/023_setup_paypal_config.rb

1 class SetupPaypalConfig < ActiveRecord::Migration
2 def self.up
3 config = Configuration.find_or_create_by_name(paypal.url)
4 config.value = https://www.paypal.com/
5 config.save!
6 end
7
8 def self.down
9 Configuration.find_or_create_by_name(paypal.url).destroy
10 end
11
12 class Configuration < ActiveRecord::Base; end
13 end

Nothing prevents me from keying off RAILS_ENV like this:


1 config.value = case RAILS_ENV
2 when development
3 https://developer.paypal.com/
4 else
5 https://www.paypal.com/
6 end
7 config.save!

An alternative would be to store the PayPal URL in a constant, and initialize the constant in config/environments/*.rb. This means much less code. But in my case, I usually have a need for a configuration-like object where admins of the system can change some values.

See all 1 articles in migration

Progress 1 articles

I have new integration tests, and they work just beautifully. I’m missing a couple more, but things are looking very good.

Thanks to Paul Watson for finding and fixing two bugs in the Git/Git case.

Finally, I have faisal which offered looking into adding SVK support for the working copy.

See all 1 articles in progress

Mendicant 1 articles

Gregory Brown, of Ruby Reports and PDF::Writer fame, wants to quit his job, sort of.

Here’s a crazy idea I just had, and I’m wondering what folks think about it.

People do open source for a lot of reasons, ranging from pragmatic to idealistic. …

What if I could just do open source for a while, non-commercially?

How much would it cost for me to do at least 80 hours a month of development on software projects such as PDF::Writer, Ruport, and some other projects I wish I had the time to get my hands on?

I did the math, and the number came out low (subjective). I could meet all my expenses and save some money for about $2000 a month. Basically, if 200 people donated $60 right now, I could take 6 months off and do nearly 500 hours of work, and that’s only if I didn’t find myself obsessed with and doing extra hours on a project.

Gregory Brown, in I’d love to quit my job!

Gregory started a Wiki where he documented his proposal. Head on over to the Ruby Mendicant for the details.

Finally, if you’d like to donate (like I did !), head on over to pledgie:

Click here to lend your support to: Ruby Mendicant and make a donation at www.pledgie.com !

See all 1 articles in mendicant

Tempfile 1 articles

This is the initial release of the Windows Tempfile Fix plugin. This plugin sets out to make Tempfile binary safe on Windows.

Installation


1 $ piston import svn://svn.teksol.info/svn/rails/plugins/windows_tempfile_fix vendor\plugins\windows_tempfile_fix

Alternatively:


1 $ ruby script/plugin install svn://svn.teksol.info/svn/rails/plugins/windows_tempfile_fix

README

As I reported on my blog at
http://blog.teksol.info/articles/2006/12/08/rmagick-and-jpeg-datastream-contains-no-image,
the Windows version of Ruby will fail to load certain files when they are used
through a Tempfile. This is because the Tempfile class does not open it’s files
in binary mode.

This plugin is very conservative. It will not install itself if it does
not detect the exact version it is supposed to be used against.

This plugin will only do it’s work if it the following expression evaluates
to true:
RUBY_PLATFORM == “i386-mswin32” && RUBY_VERSION == “1.8.4”

See all 1 articles in tempfile

Rest 1 articles

I just found Your Web Service Might Not Be RESTful If… by Paul Sadauskas. This article is a very clear and concise explanation of how to do real RESTful services.

His arguments are:

1. Present a single top-level resource;
2. All resources should be accessible by followling links from the top-level resource;
3. Don’t put API tokens in the URL or headers (use representations instead).

A very enjoyable read, and something we should all be following.

See all 1 articles in rest

Active-resource 1 articles

I have a work-in-progress branch for using Digest authentication with ActiveResource.

So, how did I do it? It wasn’t too hard actually. When I spiked, I changed ActiveResource::Connection#request to handle authentication itself. I ended up with a big mess: a new rescue clause, 10 lines of code to calculate the digest and so on. But I knew it would work. So, I git checkout . and started with tests, as it should.

The way ActiveResource is built, if a username / password is sent in, ActiveResource will send those automatically in an Authorization header, using the Basic authentication method. I need a way to turn this off. Thus grew #use_basic_authentication= and #use_digest_authentication=.

Next up, actually being able to calculate the Digest. A quick search turned up code by Eric Hodel in the form of a Ruby module: An implementation of HTTP Digest Authentication in Ruby

After a bit of cleanup and rewriting, I have a branch of ActiveResource that’s ready to be commented on. Please see francois/ar_digest and leave comments there.

An example of using Digest would be:


1 require "logger"
2 require "activeresource"
3 require "pp"
4
5 ActiveResource::Base.logger = Logger.new(STDERR)
6
7 ActiveResource::Base.site = "http://adgear.local/api"
8 ActiveResource::Base.user = "francois"
9 ActiveResource::Base.password = "my-funny-new-password-which-you’ve-never-seen-before"
10 ActiveResource::Base.timeout = 30
11
12 # Don’t attempt Basic authentication, but be sure to use Digest
13 ActiveResource::Base.use_basic_authentication = false
14 ActiveResource::Base.use_digest_authentication = true
15
16 class Site < ActiveResource::Base
17 end
18
19 pp sites = Site.find(:all)

This work was sponsored by Bloom Digital Platforms, as part of my work on their AdGear API.

See all 1 articles in active-resource

Customer support 1 articles

DreamHost had a teensy little problem with their billing system. Specifically, Josh Jones ran his biller for a couple of days in 2008, when it should have been 2007. The biller dutifully ran along and billed all the customers that hadn’t been billed yet as of December 2008.

That’s OK, I can understand things like that happen. But what I like best is this:

If this/these erroneous charge(s) by us resulted in you having any sort of overdraft/bounced check/nsf fee from your financial institution, please contact our support team from the web panel. … When we get this, we will put money on your credit card equal to the amount your bank charged you, as well as give you a DreamHost account credit for the same amount on top of that.

Josh Jones in The Aftermath

Not only will they refund the charges, but they will also credit your account for the same amount. They didn’t have to do that, but it’s nice of them to do so.

I use DreamHost for mail and file hosting / backups. I don’t use them anymore to deploy Rails applications. But this is good service.

See all 1 articles in customer support

Export 1 articles

The way I’m doing it here is obsolete, and Returning CSV data to the browser – updated has a better version.

Over on the Rails mailing list, Pete asks:

One of my apps has to export data for the backend system to process it. What’s the best way to create a CSV file in Rails and then return it as
a file to the client?

I have used two tricks in the past to export the data.

Exporting using CSV::Writer

This is simple. Use CSV::Writer and ActionController’s send_data method.

app/controllers/report_controller.rb

1 class ReportController < ApplicationController
2 def report
3 models</span> = <span class="co">Model</span>.find(<span class="sy">:all</span>, <span class="sy">:conditions</span> =&gt; [<span class="s"><span class="dl">'</span><span class="k">...</span><span class="dl">'</span></span>]) <span class="no"> 4</span> report = <span class="co">StringIO</span>.new <span class="no"> <strong>5</strong></span> <span class="co">CSV</span>::<span class="co">Writer</span>.generate(report, <span class="s"><span class="dl">'</span><span class="k">,</span><span class="dl">'</span></span>) <span class="r">do</span> |csv| <span class="no"> 6</span> csv &lt;&lt; <span class="s"><span class="dl">%w(</span><span class="k">Title Total</span><span class="dl">)</span></span> <span class="no"> 7</span> <span class="iv">models.each do |model|
8 csv << [model.title, model.total]
9 end
10 end
11
12 report.rewind
13 send_data(report.read,
14 :type => text/csv; charset=iso-8859-1; header=present,
15 :filename => report.csv)
16 end
17 end

By default, send_data will make the Content-Disposition header equal to attachment. This will ask the browser to download the file, instead of displaying it in the browser window.

Exporting using a regular view

The second method is even simpler:

app/controllers/report_controller.rb

1 class ReportController < ApplicationController
2 def report
3 @models = Model.find(:all, :conditions => [])
4 response.headers[Content-Type] = text/csv; charset=iso-8859-1; header=present
5 response.headers[Content-Disposition] = attachment; filename=report.csv
6 end
7 end

app/views/report/report.rhtml

1 Title,Value
2 <% @models.each do |model| -%>
3 "<%= model.title.gsub(‘"’, ‘""’) >",<= model.value >
4 < end -%>


Notice how I escape quotes in the view ? This is important, or else your parsing will be broken when you read the file back in. Of course, if you use CSV::Writer, you won’t have to muck with that – it will all be taken care of for you.

UPDATE 2006-03-30: Changed Content-Type from text/comma-separated-values to text/csv, and added the header optional parameter, per RFC 4180

UPDATE 2007-04-26: Just noticed that the report method above would render with a layout if one were defined in ApplicationController. Simply call #render and tell it to use no layout: render :action =&gt; :report, :layout =&gt; false

See all 1 articles in export

Patch 1 articles

This article is obsolete since the bug was corrected in core a long time ago.

Seems there’s a problem with the Rails generator. I was receiving an odd (pun intended) error:


1 $ ruby script\generate migration -tc AddGamesAndCategories1
2 odd number of arguments for Hash
3 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:130:in `[]’
4 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:130:in `add_general_options!’
5 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:130:in `call’
6 C:/ruby/lib/ruby/1.8/optparse.rb:1308:in `order!’
7 C:/ruby/lib/ruby/1.8/optparse.rb:1266:in `catch’
8 C:/ruby/lib/ruby/1.8/optparse.rb:1266:in `order!’
9 C:/ruby/lib/ruby/1.8/optparse.rb:1346:in `permute!’
10 C:/ruby/lib/ruby/1.8/optparse.rb:1373:in `parse!’
11 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:89:in `parse!’
12 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:85:in `initialize’
13 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:85:in `new’
14 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../options.rb:85:in `parse!’
15 ./script/../config/../vendor/rails/railties/lib/rails_generator/scripts/../scripts.rb:19:in `run’
16 ./script/../config/../vendor/rails/railties/lib/commands/generate.rb:6
17 C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:21:in `require__’
18 C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:21:in `require’
19 script/generate:3

After a bit of debugging, it turned out that there’s a small problem with the Rails::Generator::Options::ClassMethods#add_general_options! method.



1 opt.on(-c, —svn,
2 Modify files with subversion. (Note: svn must be in path)) {
3 options[:svn] =
4 Hash`svn status`.collect { |e|
5 e.chop.split.reverse unless
6 e.chop.split.size != 2


1 opt.on(-c, —svn,
2 Modify files with subversion. (Note: svn must be in path)) {
3 options[:svn] =
4 Hash[
`svn status`.collect { |e|
5 e.chop.split.reverse unless
6 e.chop.split.size != 2 }.flatten] }

(reformatted for visibility)

This line will return nil, unless the line splits in two (A app/models/test.rb). The problem is that if the number of lines that don’t split in two is odd, then the flattened array will contain an odd number of items, which will cause Hash.new to fail.

The solution ? compact the array before flattening. I already submitted ticket #2814 with a patch.

In the meantime, I’m running a patched copy of Rails.

UPDATE: revision 2972 applied a modified version of my original patch.

See all 1 articles in patch

Hack 1 articles

Actions with dashes

2005-10-25

This article is obsolete. You may find it’s replacement at Actions With Dashes – updated

Want to get an action with a dash in it’s name and still do something useful ? Try this:


1 class WelcomeController < ApplicationController
2 define_method("sign-up".to_sym) do
3 if request.post? then
4 # Do something
5 end
6 end
7 end

Of course, if you don’t have anything to do, just name the view file appropriately: app/views/welcome/sign-up.rhtml.

See all 1 articles in hack

Core 1 articles

A minor plugin for all of you. This plugin merely adds two new methods to Time and Date:

test/time_extensions_test.rb

1 class TimeExtensionsTest < Test::Unit::TestCase
2 def test_really_in_future
3 assert 1.second.from_now.in_future?
4 end
5
6 def test_in_future_but_past
7 assert !1.second.ago.in_future?
8 end
9
10 def test_really_in_past
11 assert 1.second.ago.in_past?
12 end
13
14 def test_in_past_but_future
15 assert !1.second.from_now.in_future?
16 end
17 end

This makes for much more readable code:


1 if @post.published_at.in_future? then
2 # do something
3 else
4 # do something else
5 end

Installation:


1 $ script/plugin install \
2 svn://svn.teksol.info/svn/rails/plugins/time_extensions \
3 vendor/plugins/time_extensions

See all 1 articles in core

Auto scope 1 articles

Ain’t Release Early, Release Often wonderful ? I just received a patch from Evgeniy Pirogov that makes the failing test from the AutoScope plugin work.

Evgeniy’s patch is available here, but you should simply svn or piston update to get the latest version, since I eagerly accepted Evgeniy’s patch.

Thank you, Evgeniy for your work.

See all 1 articles in auto_scope

Watchr 1 articles

I started using Watchr in a couple of projects. An interesting use of Watchr is to check on your tests. I have been using this script with great success so far:

test.watchr

1 def failed
2 system("growlnotify —name adgear-reporting-tests —image /Applications/Mail.app/Contents/Resources/Caution.tiff -m ‘Oops, tests failed’")
3 system("say ‘failed’")
4 end
5
6 def succeeded
7 system("growlnotify —name adgear-reporting-tests -m ‘Green tests’")
8 system("say ‘pass’")
9 end
10
11 watch((lib|test)/.+\.(js|rb)) do |_|
12 system("rake")
13 $?.success? ? succeeded : failed
14 end

Notice I’m using Mac OS X application paths, but it works just fine for me. And the regular “pass” and “failed” messages keep me abreast of what’s going on. But this style of continuous testing only works when your tests take a few seconds at most. When my tests take much longer than that, I start fiddling, opening Google Reader, checking news, whatever.

Your mileage may vary.

See all 1 articles in watchr

Andand 1 articles

For the first time ever, ragan crystallized why he didn’t like the original #andand implementation, or my own "implementation"http://blog.teksol.info/2007/11/23/a-little-smalltalk-in-ruby-if_nil-and-if_not_nil, #if_not and #if_not_nil and other implementations.

His point is that if you are writing a library, and you want to use the andand gem, you, the author of the library, will pollute the Object namespace of the application that’s using our library.

In effect, the library’s author dependencies will become dependencies of the application that’s using it, even though they don’t want or need the extensions.

The same problem happens with Mongrel and Thin : all the libraries these servers use are loaded in the application that’s running on top of them.

Thank you Reginald for clearing this up for me. I now fully understand the need for the Ruby.rewrite project.

See all 1 articles in andand

Unit 1 articles

Bob Silva posted Testing Gotchas in Rails in mid-October. I had fallen across this error myself a few times.

Well, today I had a huge code base I needed to check. I wrote the following Rake task:

lib/tasks/find_missing_fixtures.rake

1 # Find missing fixtures declarations from Rails tests
2 # Code by Francois Beausoleil (francois@teksol.info)
3 # Released in the public domain. Do as you wish.
4 desc "Finds missing fixtures declarations from your tests"
5 task :find_missing_fixtures do
6 state = :find_test_case
7 test_case = nil
8
9 Dir[test//_test.rb].each do |file|
10 File.open(file, r) do |f|
11 f.each do |line|
12 case state
13 when :find_fixture
14 case line
15 when /def test_/
16 printf "%s: %s\n", file, test_case
17 state = :find_test_case
18 when /fixtures/
19 state = :find_test_case
20 when /class (\w) < Test::Unit::TestCase$/
21 test_case = $1
22 state = :find_test_case
23 end
24
25 when :find_test_case
26 case line
27 when /class (\w) < Test::Unit::TestCase$/
28 test_case = $1
29 state = :find_fixture
30 end
31 end
32 end
33 end
34 end
35 end

Run it like this:


1 $ rake find_missing_fixtures
2 (in D:/wwwroot/wpul.staging.teksol.info)
3 test/unit/name_test.rb: NameTest
4 test/unit/application_helper_test.rb: ApplicationHelperTest
5 test/unit/application_helper_test.rb: ApplicationHelperTruncationTest

Why do I report both the file and TestCase name ? Because I put more than one TestCase per test file, as I reported in Test fixtures and behavioral testing

Of course, if I could do away with fixtures altogether…

See all 1 articles in unit

Mac 1 articles

I just got a MacBook Pro (my first Mac!) and couldn’t install many gems. All had native dependencies. All failed similarly:


1 $ sudo gem install god
2 Password:
3 Updating metadata for 1 gems from http://gems.rubyforge.org/
4 .
5 complete
6 Building native extensions. This could take a while…
7 ERROR: Error installing god:
8 ERROR: Failed to build gem native extension.
9
10 /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb install god
11 can’t find header files for ruby.
12
13
14 Gem files will remain installed in /Library/Ruby/Gems/1.8/gems/god-0.7.6 for inspection.
15 Results logged to /Library/Ruby/Gems/1.8/gems/god-0.7.6/ext/god/gem_make.out

I finally found the solution on the MacRuby Troubleshooting: can’t find header files for ruby page.

Hope this helps someone else.

See all 1 articles in mac

Ajax 1 articles

Well, I found today I needed an Ajax.Responders for something not as trivial as showing an indicator.

On the form, I have event handlers responsible for doing a PUT on each change to the field. This way, the user doesn’t need to remember to save his record when navigating away.

Since I’m working on the contact page, we need to add addresses and phones. I had it all working, except that the new address or phone’s fields weren’t AJAX-enabled.

After reading a bit, I found/remembered about Ajax.Responders. Here’s what it looks like:

public/javascripts/xl_suite/form_handler.js

1 XlSuite.FormHandler = Class.create();
2 XlSuite.FormHandler.prototype = {
3 initialize: function(root) {
4 this.root = root;
5 this.registeredFields = new Hash();
6
7 // Register an event handler to be called
8 // after each Ajax call completes
9 Ajax.Responders.register({
10 onComplete:
11 this.registerEventHandlers.bindAsEventListener(this)
12 });
13 },
14
15 registerEventHandlers: function() {
16 var editors = Selector.findChildElements($(this.root), ["input.subtleField", "select.subtleField", "textarea.subtleField"]);
17
18 for (var i = 0; i < editors.length; i++) {
19 var field = editors[i];
20 if (this.registeredFields[field.id] != field && field.id.substring(0, 4) != "new_") {
21 Event.observe(field, "focus", this.onFocus.bindAsEventListener(this));
22 Event.observe(field, "keypress", this.onKeypress.bindAsEventListener(this));
23 Event.observe(field, "blur", this.onBlur.bindAsEventListener(this));
24
25 // Remember that we processed this field
26 this.registeredFields[field.id] = field;
27 }
28 }
29 }
30
31 // rest of implementation omitted
32 );

In initialize, we prepare ourselves a Hash to remember which fields were processed, and not add multiple event handlers for the same field.

Then, in registerEventHandler (which is called on the complete event of the AJAX call), we find which fields to register event handlers on, and flag them as processed.

I didn’t find many example usages of Ajax.Responders. I hope this helps someone.

See all 1 articles in ajax

Test-driven-design 1 articles

Today, I was working on my lawn. Spreading new soil to sow new grass. I was looking at the ground, and I was pretty sure drainage wasn’t going to be good. I decided to make a test: I got out the garden hose and let it flow. Sure enough, water was pooling instead of draining. Back to the drawing board (or at least, a couple more rakes to spread things around).

As I was working the soil, a thought hit me: I’d say nearly every profession has been doing test driven design (not development) since pretty much the dawn of time:

  • the toolsmith in the ancient tribes would hit the stone, see if it did work or not, and repeat as appropriate;
  • the chef tastes his food before letting other people eat it;
  • the plumber tests the pipes before letting the water flow in;
  • the soldier tests his equipment before setting off on the battlefield;
  • naval engineers are simulation testing their latest aircraft carrier in an effort to ensure nothing is out of place;
  • certain programmers write automated tests for their code, to ensure it works as designed.

If nearly every profession has been doing it for thousands of years, why aren’t you doing it today?

See all 1 articles in test-driven-design

Google charts 1 articles

If you weren’t at Montreal on Rails 7, you really should have been there. Not because of me, but because Marc-André showed how Thin integrates with Rack, and coded a web framework in 15 minutes. Then, Julien Guimont demoed his Encrypted URL Helper. Very interesting stuff, if you need to obfuscate URLs.

Then, there was me. A quick demo of Google Charts, and this post to provide you all with the link to the mor7-google-charts-demo Git repository:

git://github.com/francois/mor7-google-charts-demo.git

Go ahead and have fun. If you make significant changes, please contact me so I can pull your changes and push them back to the public repository.

See all 1 articles in google charts

Firefox 1 articles

I’m posting this article from Firefox Beta 3. From a limited amount of testing, I feel like the speed is improved. Part of that might just be Firebug being disabled, but it’s nice anyway.

See all 1 articles in firefox

Search

Your Host

A picture of me

I am François Beausoleil, a Ruby on Rails and Scala developer. During the day, I work on Seevibes, a platform to measure social interactions related to TV shows. At night, I am interested many things. Read my biography.

Top Tags

Books I read and recommend

Links

Projects I work on

Projects I worked on