Monthly Archives: April 2011

MongoDB Honey Badger

In case you don’t know about the Honey Badger—you have to watch this video. Then you will see why MongoDB is a close cousin to this feared and fearless animal!

Developing a new project where your domain classes/tables are changing rapidly?

MongoDB don’t care!

Tired of running rake db:migrate?

MongoDB don’t care!

Need to add a new “column” to your “table?”

MongoDB don’t care!

Want to query your “table” on “columns” that don’t exist?

MongoDB don’t care!

Need to add a new index on the fly?

MongoDB don’t care!

Welcome the Nastyass MongoDB into your development lair, you won’t give a shit about your database growing and changing!

MongoDB don’t care!

Find out more about Honey Badgers here — though Randall already taught us most of the salient points!

 

The Website Development Anti-Pattern

I don’t want to come down too hard on the author, he looks like a real nice, friendly guy. Writing a post like this took time, effort, thought, and care — which I applaud.

But when I read this post on Website Development — A Practical Methodology, I had flashbacks to Waterfall-esque DoD projects. Seriously? A phased approach with no partial deliveries, no iterations, no “sneaking up on the solution?”

Is this a post from, like, 1986? A phased approach?

  • “Once everything has been appropriately documented, it is time for the Implementation Phase.”
  • “Once implementation is complete, it is time to test.”
  • “When the testing is complete… you are ready to deploy.”

I humbly submit that you might want to intersperse some of those activities into a more iterative approach to reduce the time and cost of getting things wrong and discovering it late in the phases.

I just checked the date of the post again.

It is 2011.

Someone pinch me.

MongoMapper Query Overview

There was a question on the MongoMapper Google Group from a Mongoid user about how MongoMapper handles associations. Brandon was surprised that this query returned an Array:

Product.first.releases.where(something)

Let’s break it down, one bit at a time and clear things up:

# This would be an instance of Product
Product.first # Class.

This simply gets the first element in the Array that is returned by the default “All” query on Product. Of course, without sorting, you probably would not want to do this.

# This would be a return value of an array, assuming Product <>----> * Release
Product.first.releases # Array.

In Brandon’s example, I assume “releases” is a many association. That means, an Array. Unless the association has been tweaked to have default sorting via an Association Extension, getting the “first” one might be adventurous.

# This doesn't change the above... merely adds a restrictive query clause
Product.first.releases.where(something) # Array.

Here we simply get the first element of the releases array, narrowed down by the “something” query.

Capisce?

I am not sure why, but for me it seems more logical to start my clauses with the where, and narrow them down further, or modify them… In MongoMapper, I find querying rigor is much more “loose” than say a SQL SELECT query that requires things in proper order… I would tend to write my queries in more or less this fashion:

ModelClass.where(some criteria).[sort | order | another where clause | fields | limit].[all | first | paginate]

In addition, it is important to note that MongoMapper returns a query and does not actually perform the query until you add something that needs the results. For example: all, first, paginate, sort, etc.

I can picture one of those “man page” or SQL style of fancy ways to show you how you can construct a mongomapper query given all the combinations of options for each “position” in the query…

My (unsolicited) advice is to make the query look as “natural” as possible in terms of how you might read it aloud.

Product.releases.where(:major.gt => 1).sort(:minor.desc).first # Get the latest 1.x release

(And, if the releases where clause query is common, you can create an Association Extension)

Use the Console

You can always just output the queries to the console:

>> Patient.where(:last_name=>/john/i).class
=> Plucky::Query
>> Patient.where(:last_name=>/john/i).all.class
=> Array
>> Patient.where(:last_name=>/john/i).all.count
=> 1
>> Patient.where(:last_name=>/john/i).first.class
=> Patient
>> Patient.sort(:created_at.desc).first.class
=> Patient

Association Extension

And to show an example of an extension (when you use it frequently, for example):

class Encounter
  include MongoMapper::Document
  ...
  # Associations :::::::::::::::::::::::::::::::::::::::::::::::::::::
  many :events, :limit => 30, :order => 'msg_timestamp desc' do
    ...
    def images
      where(:type => [EventConstants::EventType.to_text(EventConstants::EventType::IMAGE)]).order(:created_at.desc).all
    end

    def charts
      where(:type => [EventConstants::EventType.to_text(EventConstants::EventType::ED_SUMMARY)],
            :file_version.in => ["P", "F"]).order(:created_at.desc).all
    end

    def admits
      all(:type => [EventConstants::EventType.to_text(EventConstants::EventType::ADMIT)])
    end
  end
  ...
end

# For a given encounter
enc=Encounter.find('4dadad188951a20727000160')
>> enc.events.images.count
=> 7
>> enc.events.images.class
=> Array
>> enc.events.images.first
=> #

Named Scope

If you will need dynamic querying, you could use a Named Scope as follows:

scope :by_days_old,  lambda { |age| where(:msg_timestamp.gt => age.days.ago) }

This can be used as follows:

Encounter.by_days_old(10)
=> #Fri Apr 15 03:35:53 UTC 2011}>

Looking at Gem Code is Easy with GemEdit

Ever want to pop into the source code of some gem that you are using?

Gem Edit is a nice and easy way to do this (if you aren’t using RubyMine, for example):

[sudo] gem install gemedit

In the terminal, simply type:

[develop*]$  jonsmac2-2:source jon$  gem edit -e mate mongo_mapper

And you will see:

Found gem for 'mongo_mapper' with version >= 0
Opening the following gems with mate:
  mongo_mapper-0.8.6 /Library/Ruby/Gems/1.8/gems/mongo_mapper-0.8.6
Running `mate "/Library/Ruby/Gems/1.8/gems/mongo_mapper-0.8.6"`

and Voila, Mongo Mapper appears in TextMate:

Mongo Mapper

Mongo Mapper Source Code

In an IDE, you can often get to the source code for a single method, but not as “completely” as with gemedit and TextMate, IMHO.

 

Mongo Mapper from IDE

Mongo Mapper from IDE

 

@martinstreicher gave me a good tip when using bundler (which I use on my Rails 3 apps):

export EDITOR=mate;bundle open <gem>

Factory Girl and MongoMapper

You were probably hoping for some Rosey the Riveter poster…

Factory Folder

Factory Folder

Instead, I am going to extend my small MongoMapper example to include Factory Girl. The steps are pretty simple:

  1. Go here to install…
  2. Create your factories
  3. Use the factories in Cucumber/RSpec

Factory Construction

I created a new “factories” folder under the spec folder:

The factories for User and Event are quite simple:

Factory.define :user do |u|
  u.name ('a'..'z').to_a.shuffle[0..7].join.capitalize
end

and

require 'factory_girl'
def dummy_word(len=6)
  ('a'..'z').to_a.shuffle[0..len].join.capitalize
end

def dummy_date
  secs_in_day = 24*60*60
  Time.now + (rand(60)*secs_in_day - 30)
end

Factory.define :event do |e|
  e.title "#{dummy_word} #{dummy_word 3} #{dummy_word 10}"
  e.date  dummy_date
end

Refactor Original Setup

Instead of using this style of test data creation:

@event = Event.create(:title => "Code Retreat Timbuktoo", :user => @fred)

We will use the new factory as follows:

@event = Factory(:event, :title => "Code Retreat Timbuktoo", :user => @fred)

Refactor Cucumber

The given went from this:

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => fred)
  end
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => harry)
  end
  Event.count.should == 20
end

to this – including refactoring out dummy_title, and reducing it to one loop:

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    evt = Factory(:event, :title => dummy_title,
                          :date  => dummy_date,
                          :user  => fred)
    evt = Factory(:event, :title => dummy_title,
                          :date  => dummy_date,
                          :user  => harry)
  end
  Event.count.should == 20
end

Subtle Details

The beauty of having tests is that I could easily mess around with getting some of the Factory Girl configuration stuff in the right place. Try something, run the test, adjust as needed until all are back to green.

The file features/support/env.rb got some additions so that Cucumber could find the factories:

$LOAD_PATH << File.expand_path('../../../app/model' , __FILE__)
require 'user'
require 'event'
require 'spec/factories/events.rb'
require 'spec/factories/users.rb'
load 'config/mongo_db.rb'

All the tests still pass!

More Complicated Example

For a project I work on, my factories look like this, with auto-creation of random IDs:

def random_months(months)
  day_in_secs = (24*60*60)
  (1+rand(months))*30*day_in_secs
end

# ----------- GROUP -----------
Factory.sequence :group_num do |n|
  "99#{n}#{rand(n)}"
end

Factory.define :group do |g|
    g.group_num {Factory.next(:group_num)}
    g.name "Greatest Group"
end
# ----------- ACCOUNT -----------
Factory.sequence :doctor_num do |n|
  "999992#{n}#{rand(200+n)}"
end

Factory.sequence :login do |n|
  "AB#{rand(n*68)}bx#{rand(200+n)}"
end

Factory.sequence :msid do |n|
  "CQ987Z12#{n}#{rand(n)}"
end

Factory.define :account do |a|
  pw = 'password'
  a.msid { Factory.next(:msid) }
  a.doctor_num { Factory.next(:doctor_num) }
  a.first_name "James"
  a.last_name "Jones"
  a.role 'user'
  a.password pw
  a.password_confirmation pw
  a.email Setting.get("AutoEmail")
  a.login { Factory.next(:login) }
end

# ----------- PATIENT -----------
Factory.sequence :patient_num do |n|
  "#{n}#{rand(300+n)}"
end

Factory.define :patient do |pt|
#  pt.patient_num "10000009"
  pt.patient_num {Factory.next(:patient_num)}
  pt.emr_num "1853286"
  pt.first_name "John"
  pt.last_name "Johnson"
  pt.dob {(Time.now - random_months(36))}
  pt.count_public_encounters 1
  pt.count_public_events 2
end

 

Cucumber, RSpec, MongoMapper, Git, Oh My!

Nick left a question on my many-to-many associations post. He wanted to know more about sorting by date and querying…

So I decided to (over achieve and) show how I would approach that as if I were adding a new feature:

  • Adding a “date” key field and an index
  • Using Cucumber to drive the new feature from the desired behavior à la BDD.
  • Querying with a date sort tacked on…

You can follow my progress from the earlier version to this one by examining the commit history:

Commit History

Commit History for Adding Dates to the Event Class

Looking at the “added initial event date functionality” commit, you can see how I added some new files to allow for Cucumber and MongoMapper:

Commits for Event Dates

Commits for Event Dates

For Cucumber, I added the “/features” stuff.

And since I wanted to start testing the code, I had to make the database functional, so I added “mongo_db.rb” – and I assume you have MongoDB installed and running locally.

##### MONGODB SETTINGS #####
MongoMapper.connection = Mongo::Connection.new('localhost', 27017, :pool_size => 5)
MongoMapper.database = "event-development"

I am never quite sure if there is a “perfect” way to wire up small, non-Rails apps like this one to use MongoDB. But what I have done works good enough to allow for a simple example to run.

BDD Cycle

So I began the BDD cycle by creating a “feature branch” and switching to a new branch from the current master branch:

git checkout -b event_dates master

Next I wrote the feature for the new behavior:

Scenario: Sort Events by Date
  Given A set of events
  When I display the events
  Then I should see them sorted by latest date first

When you run Cucumber, you will get the default code for steps – all pending of course.  Naturally, I did one step at a time, to take each one from “pending” to green. Working on the Given, then the When, and finally the Then, I came up with these steps:

Given

Here I wanted to generate a set of data so that we could see if the list was sorted properly. You can check out the code on github for the randomness baked into the “dummy_*” helper methods. And I wanted to create the events for two users.

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => fred)
  end
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => harry)
  end
  Event.count.should == 20
end

When

Sort of playing along as if this is a web request, I coded this step to return a “response” that is generated by the “list all events” class method. In true BDD fashion, this is the code I wish I had 🙂

When /^I display the events$/ do
  @response = Event.list_all
end

Then

The proof is in scanning the resultant “response” object to ensure date order is correct:

Then /^I should see them sorted by latest date first$/ do
  first = Date.parse(@response.third[0..10])
  last = Date.parse(@response.last[0..10])
  first.should > last
end

Outside – In

As soon as I ran the “When” I got a failure due to Event.list_all not existing. So off to the RSpec-land we go, to write the expectations for list_all. This is known as the “Outside-In” approach. (I learned this term from the excellent RSpec book, and it looks like you can watch a video about it here.)

The behavior expressed above (in Cucumber) can be thought of as more of an “outer,” acceptance/integration style of test. Typically it would be the User Interface (UI) – but I have been known to blur that line, since not all code is about UI and since Cucumber is so darn fun to use. Working at this outer level often leads to expressing what is expected of our actual code; in this case, that Event have a class-level method that returns a list of it’s instances (a.k.a., documents). Since we are talking about the behavior of a class, that is more of the “inside” of the application. Not something that an external user might care so much about directly, but rather something that supports the end behavior in an indirect fashion. For the “inside” we turn to RSpec (basically a better-than-unit-test, unit test tool).

  • Outside ≈ Feature ≈ Cucumber
  • Inside ≈ unit test ≈ RSpec
  describe "#list_all" do
    it "should show each event, ordered by date" do
      response = Event.list_all
      response.should_not be_empty
      response.class.should == Array
      response.size.should == Event.count + 2 #for title and column header
      # Yes, you should not output stuff as part of your tests, but this *is* our UI 🙂
      puts response
    end
  end

Many times, my initial pass at a new method is to simply return what is expected. Then write another test to make that fail. Sort of “sneak up on the answer.” But here it was easy enough to simply output some real text from the get-go:

  def self.list_all
    response = []
    response << "%s %s %s" % ["*"*10, "LIST OF EVENTS", "*"*10]
    response << "%6s %15s               %s" % ["Date", "TITLE", "Attendees/Interested/Likes"]
    events = Event.all(:order => 'date desc')
    events.each {|e| response << e.to_summary}
    response
  end

Oops. LOL (:-D) While writing this post, I found a mistake when testing my code a bit further than my initial commit.

I decided to remove the order part of the query, which revealed that the Cucumber feature still passed. Crap! So, it wasn’t so easy after all! Dope.

    events = Event.all

Second Attempt

My initial way of generating records resulted in the documents magically being in the right date order by default. Tests passed, but the test was wrong – not vigorous enough testing!

Note to self:
no matter how trivial things seem, write failing tests
that contradict each other – so to speak.

So, I tweaked the document generator to better randomize the list of events such that we won’t accidentally have them all in proper order by default:

  ...
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => Time.now + (rand(60)*secs_in_day - 30),
                 :user => fred)
  end
  ...

And, instead of just spot-checking the order, here is a new RSpec test to ensure each event is in proper order, date-wise:

    it "should show each event, ordered by date" do
      response = Event.list_all
      response.should_not be_empty
      response.class.should == Array
      response.size.should == Event.count + 2 #for title and column header
      r_prior_date = Date.parse(response[2][0..10])
      response[3..response.size].each do |r|
        date = Date.parse(r[0..10])
        date.should < r_prior_date
        r_prior_date = date
      end
      # Yes, you should not output stuff as part of your tests, but this *is* our UI 🙂
      puts response
    end

Now we’re talking! A failed test:

'Event#list_all should show each event, ordered by date' FAILED
expected: < Wed, 20 Apr 2011,
     got:   Wed, 20 Apr 2011

And similarly, I re-wrote the Cucumber test. Funny thing, further testing revealed that the error above was not actually a legitimate fail as it turns out! I discovered I needed “<=” instead of just “<“– sometimes the simplest things aren’t so simple after all. Especially when it comes to setting up sample data.

Then /^I should see them sorted by latest date first$/ do
  last_date = Date.parse(@response.third[0..10])
  @response[3..@response.size].each do |r|
    date = Date.parse(r[0..10])
    date.should <= last_date
    last_date = date
  end
end

And I got the above test to fail by “stepping back” and removing the “order by” clause to get me back to an original, non-sorted listing. Good! Now we can step forward again and try to get the functionality that we are looking for to work.

Cucumber Failing Tests

Cucumber Failing Tests

I re-enabled the order clause to see if the tests would now pass:

events = Event.all(:order => 'date desc')

And, fortunately, the tests are indeed passing:

Cucumber Passing Tests

Cucumber Passing Tests

Commit on Green

Once you get the bits of functionality working, commit (even if you still have pendings). Committing locally has no downside 🙂 Here I will commit and push to the repo (the “$” is my prompt (well, not really), and the #comments are not part of the command line!):

$git status #You can see your changes
$git commit -a -m "added initial event date functionality"  #commit your changes
$git checkout master #switch to the master branch
$git merge --no-ff event_dates #merge all of your local feature branch commits, preserving each
$git push origin master #Pump it up to the repo
$git branch -d event_dates #Get rid of the feature branch

This rhythm gets to be very familiar.

More on Querying

You may have noticed some of the queries above, and this was one of Nick’s questions…

With MongoMapper, you can chain Plucky queries as follows:

  def self.list_all(a_user=nil)
    response = []
    response << "%s %s %s" % ["*"*10, "LIST OF EVENTS", "*"*10]
    response << "%6s %15s               %s" % ["Date", "TITLE", "Attendees/Interested/Likes"]      events = nil     if a_user.nil?       events = Event.all(:order => 'date desc')
    else
      events = Event.where(:user => a_user).all.sort(:date.desc)
    end
    events.each {|e| response << e.to_summary}
    response
  end

I added a new feature that shows off the above query, quickly ran through the entire process again, from git checkout to git push, with Cucumber and RSpec and code in between. You can find it all in the source code.

Cucumber Show Events For User

Cucumber Show Events For User

Git is really an amazing revision control tool… I can’t imagine using anything else now. Here is an example of looking at the “Network Graph” of my little project:

github network graph

Github Network Graph

Pow — A Great Server for your Mac Bat Cave

Somehow I stumbled across this great little tool  — oh yeah, John Nunemaker mentioned “powder,” which was “Syntactic sugar for http://pow.cx/

Huh? Syntactic what? For “pow” what? John hasn’t steered me wrong yet, so I decided I needed to look at what Pow was. In 37Signals own words:

Pow: Zero-configuration Rack Server for Mac OS X

In other words:

You can run a bunch of your rails apps all at once, with ease.

This is worth the effort to try and get running to see if it will help ease how you work with multiple Rails apps.

Since my experience was a bit off from the awesome-looking Screencast teaser, I figured I would share it in case others might benefit from the troubleshooting.

Once I solved the Pow-not-working-like-magic-from-the-start problems, it worked right out of the box with a Rails3 App. Not so much with my Rails2 apps. So, you can either try to do it the Rails3 way from the start (see below for instructions), or barge ahead and see how far you get (knowing now what I do for Rails2, barging is probably a pretty safe bet).

Yes, I am repeating what is in the main website… but then aggregating common troubleshooting experiences ad fixes below, to save you (and my colleagues) time.

Install Pow:

$ curl get.pow.cx | sh

Be certain you see “*** Installed” or it is not! (See Troubleshooting section below if need be.)

jonsmac2-2:.pow jon$ curl get.pow.cx | sh
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100  2722  100  2722    0     0  17322      0 --:--:-- --:--:-- --:--:-- 34455
*** Installing Pow 0.2.2...
*** Installing local configuration files...
*** Starting the Pow server...
*** Installed

If you get upset, just uninstall:

$ curl get.pow.cx/uninstall.sh | sh
Symbolic Link to Rails3 App

Symbolic Link to Rails3 App

Configure:

$ cd ~/.pow
$ ln -s /path/to/myapp

To the right is an example. Name the link if you do not want to use the default folder name. For example:

ln -s ~/railsprojects/first_app jkfirstapp

This allows you to then enter the following into the browser, using the symbolic link name (or the default folder name):

http://jkfirstapp.dev

If that didn’t work, continue onward, your journey will be short (I hope).

Troubleshooting

I was disappointed when the magic didn’t happen right away. Boy does the Ruby community spoil us! So a hunting I will go…

Create a Rails3 App

If you are only doing Rails2, see below for the config.ru fix. Otherwise, creating a vanilla Rails3 app is a good way to test that Pow is properly configured. Here is a simple Rails3 app (from Michael Hartl’s Ruby on Rails3 Tutorial):

mkdir ~/railsprojects
rails new first_app

edit the Gemfile:

gem 'rails', '3.0.1'
gem 'sqlite3-ruby', '1.2.5', :require => 'sqlite3'

Then execute:

bundle install
rails server

It should be running with the default server page.

I got Nothing!

If pow did not install, or you get seemingly NOTHING when going to the browser webpage, look no further than here. The following worked for me (and others). Seems that the permissions are not quite right sometimes:

before:

-rw-rw-r--   1 jon  staff   518B Apr 18 13:42 cx.pow.powd.plist

So I ran this:

jonsmac2-2:.pow jon$ launchctl load -Fw ~/Library/LaunchAgents/cx.pow.powd.plist
jonsmac2-2:.pow jon$ chmod 644 ~/Library/LaunchAgents/cx.pow.powd.plist
jonsmac2-2:.pow jon$ curl get.pow.cx | sh

And, after the re-install, the perms ended up as you might expect:

-rw-r--r--   1 jon  staff   518B Apr 18 13:42 cx.pow.powd.plist

Checking It’s Running

This is admittedly odd output for a “working” install, but it gives you at least one data point to compare what you get versus this. Do the following:

cd ~/Library/Application Support/Pow/Current/bin
jonsmac2-2:bin jon$ ./pow

On my Mac it results in:

node.js:134
    throw e; // process.nextTick error, or 'error' event on first tick
    ^
Error: EADDRINUSE, Address already in use
at HttpServer._doListen (net.js:1089:5)
at net.js:1060:14
at Object.lookup (dns.js:159:5)
at HttpServer.listen (net.js:1054:20)
at Array.<anonymous> (/Users/jon/Library/Application Support/Pow/Versions/0.2.2/lib/daemon.js:34:27)
at EventEmitter._tickCallback (node.js:126:26)

Domain Isn’t Setup

If you see this:

No such app

No such application can be found!

Then you know you have screwed up the symbolic link.

Try again, and be careful and exact.

Trouble Accessing Dev domains

Some folks needed this simple fix:

touch /etc/resolver/dev

Rails2: Cannot GET /? –> config.ru is Missing!

That’s right, you see: Cannot GET / in the browser. Bet you never saw that before!

This means that you are missing the Rackup config.ru file in your application’s root. Try adding this file (you can see it in the folder screenshot above):

# This file is used by Rack-based servers to start the application.
require "./config/environment"
use Rails::Rack::LogTailer
use Rails::Rack::Static
run ActionController::Dispatcher.new

Pow Can’t Start Your App

This is most likely, genuinely, truly your error… Sorry to say!

The error below is because the config.ru was completely wrong (I was trying the Rails3 project file (duh — it invokes a class in a module named after the app itself)):

Pow can’t start your application.

/Users/jon/railsprojects/track_my_league/tml raised an exception during boot.
NameError: uninitialized constant FirstApp
/Users/jon/.gem/ruby/1.8/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:443:in `load_missing_constant'
/Users/jon/.gem/ruby/1.8/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:80:in `const_missing'
/Users/jon/.gem/ruby/1.8/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:92:in `const_missing'
/Users/jon/railsprojects/track_my_league/tml/config.ru:4
/Users/jon/Library/Application Support/Pow/Versions/0.2.2/node_modules/nack/lib/nack/builder.rb:4:in `instance_eval'
/Users/jon/Library/Application Support/Pow/Versions/0.2.2/node_modules/nack/lib/nack/builder.rb:4:in `initialize'
/Users/jon/railsprojects/track_my_league/tml/config.ru:1:in `new'
/Users/jon/railsprojects/track_my_league/tml/config.ru:1

Here the error is that I do not have the gem environment set up properly with RVM (using .rvmrc). Easily diagnosed because I can’t even start the app manually!

SystemExit: exit
./config/boot.rb:66:in `exit'
./config/boot.rb:66:in `load_rails_gem'
./config/boot.rb:54:in `load_initializer'
./config/boot.rb:38:in `run'
./config/boot.rb:11:in `boot!'
./config/boot.rb:110
./config/environment.rb:7:in `require'
./config/environment.rb:7
/Users/jon/railsprojects/tml/TML/config.ru:3:in `require'
/Users/jon/railsprojects/tml/TML/config.ru:3
/Users/jon/Library/Application Support/Pow/Versions/0.2.2/node_modules/nack/lib/nack/builder.rb:4:in `instance_eval'
/Users/jon/Library/Application Support/Pow/Versions/0.2.2/node_modules/nack/lib/nack/builder.rb:4:in `initialize'
/Users/jon/railsprojects/tml/TML/config.ru:1:in `new'
/Users/jon/railsprojects/tml/TML/config.ru:1

Hope this helps!

Multiple Many-to-Many Associations in MongoMapper

There was a question in the Google Group for MongoMapper, so I decided to post an answer in the form of a simple demo. You can find the source code on Github.com here.

The basic shape of the problem was this:

Users Sponsor and Attend Events

Users Sponsor and Attend Events

And the solution issues were around the multiple many-to-many associations, more or less.

For a simple one-to-many, MongoMapper has the normal:

  • User has many :events
  • Event belongs_to :user (its Owner)

But how to do the other associations? A given User can be involved with many events in different capacities:

  • Attending
  • Interested in attending
  • Likes

There are different ways to tackle these many-to-many associations.

  1. You can use a Set (to obtain the uniqueness factor) of Users that are attending or are interested.
  2. You can use an Array of instance IDs (I think this is probably the more standard technique)
class Event
  include MongoMapper::Document

  key :title, :required => true

  key :user_id
  belongs_to :user

  # One way to do it...
  key :attendees, Set
  key :interested, Set

  # Another way to do it...
  key :like_ids, Array
  many :likes, :class_name => 'User', :in => :like_ids

  def attending(a_user)
    # self.push_uniq(:attendees => a_user.id)
    attendees << a_user.id
    save
  end

  def interested_in(a_user)
    interested << a_user.id
    save
  end
...

In which direction you allow making the association, that is up to your application’s needs. For example, above you can see that an Event instance could be messaged with the user to indicate attending or interested_in. The “likes” is immediately accessible from an Event, or I could have added a wrapper method (def likes(a_user)).

And the User class has some simple retrieval methods to see what a User likes, what they are attending, and what they are interested_in:

class User
  include MongoMapper::Document

  key :name, :required => true

  many :events

  def likes
    Event.where(:like_ids => id).all
  end

  def attending
    Event.where(:attendees => id).all
  end

  def interested_in
    Event.where(:interested => id).all
  end

  def likes_event(event)
    event.likes << self
    event.save!
  end

To see how the different styles are used, you can check out the specs. For example:

Adding users who like an Event:

  it "should track interested" do
    expect {
      @event.interested_in(@jared)
      @event.interested_in(@sally)
    }.to change {@event.interested.size}.by(2)
  end

it "should allow 'likes'" do
    expect {
      @event_2.likes << @martha
    }.to change {@event_2.likes.size}.by(1)
  end

Or from the User perspective:

  it "should allow me to add an event I like" do
    @fred.likes_event(@event_2)
    @event_2.likes.size.should > 0
    @fred.likes.count.should > 0
  end

Or attendees, from the Event:

  it "should list the events I am attending" do
    [@fred, @harry].each {|u| @event.attending(u)}
    @fred.attending.count.should > 0
  end

RVM Install Troubleshooting

All I wanted to do was sit down with Mike Hartl’s “Ruby on Rails 3 Tutorial” yesterday. Hours later, with nothing to show other than a zillion open terminals, I punted and watched the Flyers beat the Sabres despite playing like girls (I know, unfair to girls!). Then I went to bed.

So I must have had a dorked up installation of RVM or something, because when I tried to install Ruby 1.9.2 and Rails 3 and upgrade RVM I ran into <ahem> “issues.” I was getting weird errors like:

error installing rails i18n requires rubygems version 1.3.5
gem_runner.rb:85:in `<top (required)>': undefined method `load_plugins'

After exploring all sorts of similar woes that other people had and not really getting that magic googullet (Google Bullet), I decided to go all nuclear (apologies to Japanese readers). I was able to run:

rvm implode

Which made “rvm -v” fail to find rvm — a good thing. It meant I had uninstalled RVM. (Or so I thought!)

I also deleted the RVM stuff from the bash_profile file(s). Just to attempt to eliminate voodoo dolls.

But I still ran into stupid errors (stupid voodoo dolls!):

error installing RVM fatal: Not a git repository (or any of the parent directories): .git

After much pain and suffering (well, only in a wimpy software relative sense), my guess is that I installed rvm as sudo at one point or installed the gem, or somehow had things “cross-eyed.” I don’t know…

I guess I should be more diligent, because in general you get spoiled by Ruby and gems and Rails and the general euphoria of how easy things are. Until they aren’t.

Today, I ended up fixing my system by looking for “rvm” on my system, and cleaning up leftovers. The crumbs from my sloppy eating at the trough of Ruby, I suppose.

I saw some directories holding rvm docs. I whacked them.

I saw some directories with rvm source, archives… Gone.

I uninstalled the rvm gem. (Oops. Who put that there?)

Finally, I retried the install:

jonsmac$ bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

And added this to ~/.bash_profile And I restarted terminal because I always forget how to do that source ~/.bash_profile thing…

jonsmac2-2:~ jon$ rvm -v
rvm 1.6.2 by Wayne E. Seguin (wayneeseguin@gmail.com) [https://rvm.beginrescueend.com/]

Shazam! No errors. Thanks Wayne!

Now I can get past page 15 in Mike’s tutorial book!

Ruby Metaprogramming in the Small

I have been thoroughly enjoying working with Ruby this past year (thanks Lee!). However, only recently have I been getting brave/comfortable/wise enough to try out some metaprogramming. Okay, so maybe I am a little slow… The scourge of making deadlines and releasing software meant I sometimes just had to give up trying to get to an elegant solution that I thought was possible, but was unable to make work in the time allotted. I try to be pragmatic, if nothing else.

But here is a little example of how easy it is to exploit Ruby’s omniscient metaprogramming system.

Background

The project is a Rails app using MongoMapper. I needed to enhance the way we create User Accounts to accommodate importing a user from a CSV file (dumped from the hospital’s account management system).

First up: “Insert New Record” — which went pretty smoothly. Next, I wanted to permit merging import data with an existing account. (For example, changing the last name when married status changes.)

Round 1: Get it to work

Using a TDD approach and RSpec to flesh out the low-level class behavior, I “snuck up on the answer,” one small test at a time. I tend to use a “get it to work with brute force” approach at the outset. Leading to sometimes bulky code as can be seen below. So, one-by-one, I kept adding to the merge code for each new attribute that would allow updating.

NOTE: I added some (all?) of the code comments below for this blog post. Plus I took liberties to not show everything…

  class Account
    include MongoMapper::Document
    # Rest of class omitted
    def self.create_or_merge(fields)
      raise ArgumentError if fields.nil?

      # Some code omitted

      # There are 2-3 legal ways to identify a unique account 🙁
      account = Account.find_by_identifiers(login, msid, doctor_num)

      if account.nil?
        # Create
        if login.blank?
          login = generate_login(fields[:first_name], fields[:last_name], msid, doctor_num)
          fields[:login] = login
        end
        account = Account.create(fields)
        account.save!
      else
        # Merge
        first_name = fields[:first_name]
        last_name  = fields[:last_name]
        phone      = fields[:phone]
        email      = fields[:email]
        doctor_num = fields[:doctor_num]

        account.last_name = last_name unless last_name.blank?
        account.first_name = first_name unless first_name.blank?
        account.phone = phone unless phone.blank?
        account.email = email unless email.blank?
        account.save!
      end
      account
    end

  end

Round 2: Extract Method

The first step to making “create_or_merge” simpler was to yank out the blob of merge code into it’s own method. So the logic in create_or_merge looks a bit cleaner:

  • If can’t find account,
    • create new one;
  • else
    • merge this data into existing account.
  class Account
    include MongoMapper::Document
    # Rest of class omitted
    def self.create_or_merge(fields)
      raise ArgumentError if fields.nil?
      # Some code omitted

      # There are 2-3 legal ways to identify a unique account 🙁
      account = Account.find_by_identifiers(login, msid, doctor_num)

      if account.nil?
        #Create
        if login.blank?
          login = generate_login(fields[:first_name], fields[:last_name], msid, doctor_num)
          fields[:login] = login
        end
        account = Account.create(fields)
        account.save!
      else
        #merge
        account.merge(fields)
      end
      account
    end
    def merge(fields)
      puts "Merging #{fields.inspect}"
      first_name = fields[:first_name]
      last_name  = fields[:last_name]
      phone      = fields[:phone]
      email      = fields[:email]
      doctor_num = fields[:doctor_num]

      self.last_name = last_name unless last_name.blank?
      self.first_name = first_name unless first_name.blank?
      self.phone = phone unless phone.blank?
      self.email = email unless email.blank?
      self.doctor_num = doctor_num unless doctor_num.blank?
      save!
    end
  end

Round 3: Introduce dynamic method calls

It is plain to see the repeating nature…based on the fields being passed in:

self.KEY = VALUE unless VALUE.blank?

If only there were a way to not have to write out repeating lines of code for each attribute we need to merge. Well, enter the ability to invoke instance methods by name, and passing in parameters:

Normal:

self.last_name = last_name unless last_name.blank?

Metaprogramming:

self.send("last_name", "Franklin")

And getting the name from the fields hash

self.send("#{k.to_s}=", v)  unless v.blank?

Also, there is a need to tailor which fields are allowed to be merged, or overwritten; hence, the introduction of this bit of “allow_overwrite” complexity.

  def merge(merge_fields)
    allow_overwrite = [:first_name, :last_name, :doctor_num, :phone, :email]
    # Only merge fields that are permitted... tossing out any illegal fields
    fields = merge_fields.each {|k,v| merge_fields.delete(k) unless allow_overwrite.include?(k.to_sym)}
    fields.each_pair do |k,v|
      # Update the field with new data, if available
      self.send("#{k.to_s}=", v)  unless v.blank?
    end
    save!
  end

Round 4: Compress slightly, removing one iteration loop

Instead of looping twice through the fields, I reduced it to a single pass.

  def merge(merge_fields)
    allow_overwrite = [:first_name, :last_name, :doctor_num, :phone, :email]
    merge_fields.each do |k,v|
      next unless allow_overwrite.include?(k.to_sym)
      self.send("#{k.to_s}=", v)  unless v.blank?
    end
    save!
  end

The Tests

Here is a snippet from my RSpec tests:

  # Uses metaprogramming too...
  def check_merging(field, new_value)
    @fields[field.to_sym] = new_value
    expect {
      account = Account.create_or_merge(@fields)
    }.to_not change { Account.count }.by(1)
    act = Account.find(@account.id)
    act.instance_eval(field).should == new_value
  end

  describe "being merged" do
    before do
      @group_num = "009015"
      group_name = "Country Doctor Pediatrics"
      grp = Group.find_by_group_num_or_create(@group_num, group_name)
      @doctor_num = "6709#{rand(20)}"
      @fields = {:login      => "jmadison",
                 :email      => "johns@CountryPedDocs.com",
                 :doctor_num => @doctor_num,
                 :name       => 'Dr. John Madison',
                 :first_name => "John",
                 :last_name  => "Madison",
                 :group_name => group_name,
                 :group_num  => @group_num
      }
      @account = nil
      expect {
        @account = Account.create_or_merge(@fields)
      }.to change{ Account.count }.by(1)
    end

    it "should merge new last name" do
      new_value = "Mattson"
      field = "last_name"
      check_merging(field, new_value)
    end

    it "should merge new first name" do
      new_value = "Mary Lou"
      field = "first_name"
      check_merging(field, new_value)
    end

    it "should merge new phone" do
      new_value = "123-321-1234"
      field = "phone"
      check_merging(field, new_value)
    end

    it "should merge new email" do
      new_value = "some_good_email@humptyfratz.biz"
      field = "email"
      check_merging(field, new_value)
    end

    it "should merge new doctor_num" do
      new_value = @doctor_num.reverse
      field = "doctor_num"
      check_merging(field, new_value)
    end

    after do
      if @account
        @account.destroy
        @account.save
      end
      Account.hard_delete
    end
  end