Category Archives: testing

Prefer Explicit Cucumber Tests

An Inexact Test

Innocently enough, one of our developers added this feature, which passes:

  Scenario: Create Field
    When I manage the "acme" org
    And I go to the user field management list for "acme"
    And I click "New"
    And I set the following values
      | Name     | Test Field |
      | Required | true       |
    And I click the "Create" button
    Then I should see "Test Field"
    And I should see "test_field"
    And I should see "Yes"

Note: for now I am not going to discuss the other improvements that could be done with this feature (e.g., to remove the “click” steps).

Unfortunately, the actual page looked as follows (using save and open page):

Name        Required?  Key 	
First name  Yes        first_name
Last name   Yes        last_name
Test Field  No         test_field

Sure enough, the text “Yes” was indeed on the page, given the step was defined simply as shown below. However, the intent of the test was actually not correctly followed in the code.

Then(/^I should see "(.*?)"$/) do |text|
  expect(page).to have_text(text)
end

An Explicit Test

So one way to ensure the proper data appears in the proper place on the page, especially when it is in a table, is to do the following.

Add an ID to the page location:

    %table
      %thead
        %tr
          %th.text-center Name
          %th.text-center Required?
          %th.text-center Key
          %th.text-center
      %tbody
        - user_fields.each do |field|
          %tr{id: dom_id(field)}
            %td.text-center= field.name
            %td.text-center
              -if field.required
                Yes
              -else
                No
            %td.text-center= field.key

Change the feature to expect the text exists in the context of the record:

  Scenario: Create New Field
    Given I follow "User Fields" in the sidebar
    And I follow "New Field"
    And I set the following values
      | Name     | Zodiac Sign |
      | Required | true        |
    And I click the "Create" button
    And I should see the following user field:
      | name        | required | key         |
      | Zodiac Sign | Yes      | zodiac_sign |

And define a step something like this:

Then(/^I should see the following user field:$/) do |table|
  # table is a table.hashes.keys # => [:name, :required, :key]
  field = UserField.find_by(name: field_name)
  dom_id = "user_field_#{field._id}"
  table.hashes.each do |row|
    dom_id = dom_id_for_field(row[:name])
    within("tr##{dom_id}") do
      expect(page).to have_text(row[:key])
      expect(page).to have_text(row[:required])
    end
  end
end

Test Your Test!

I also like to check my tests! That is, go to the code where the text “Yes” is defined, and change it to something else. If your tests still pass, you know your test is flawed!

Summary

By adding a “findable” id to the output, you can more exclusively assert some text is on the page in the correct location!

Other approaches

  • ensure you eliminate the other non-essential elements from the page, so there are no “false positives”
  • Check for a very bizarre string that will not accidentally appear, like “Zebra Fangs”

 

uTest Interview

I “ran” into the nice folks at uTest, and they asked me a handful of questions.

I answered… in a shockingly (for me) succinct way.

1. As a long-time agile coach, you can probably tell right away if agile is going to be employed successfully within a company or organization. If you had to pick one quality or trait that’s required for agile success, what would it be? In other words, what’s the first thing you look for when beginning a coaching project?

[JK] Willingness to change. That’s all I ask. Be open-minded to trying things a different way.

2. Looking back, did you ever think the agile movement would grow to where it is today? What’s surprised you the most about agile’s course over the last decade? The good and the bad.

[JK] No. How could 4 measly bullet points cause so much ruckus?! The biggest problem I see is the co-opting of the term “agile.” That is, folks are doing agile in name only. They don’t really get the subtle nuances about what it means to be agile, and simply go through some motions and try to “do” agile. While learning by doing is a key technique for learning anything new, somehow, many people seem to just do a handful of activities without much reflection or introspection.

3. Fill in the blank: The most common agile mistake development teams make is ____.

[JK] Not thinking. Agile requires continuous use of thinking… Are we improving? Will this help? Should we stop doing this activity? Should we do more of this activity? It takes effort to avoid complacency, which is hard for most of us.

4. From what we read, much of the inspiration for the agile movement originated not in the software space, but rather in the production/manufacturing space. How often (if at all) do you consult on non-software projects and how does it change the way agile is applied?

[JK] I’m not so sure that is true of the original 17 co-authors. While many of the lean/kanban concepts popular today owe their theories to manufacturing/production processes, I don’t recall any of the original folks waxing eloquent about being inspired by some non-software gigs. But I could be wrong. I have consulted on manufacturing automation/tracking/planning processes – and for that, a lot of the agile techniques apply. But I mostly focus on software projects.

5. A previous guest of our blog once referred to the “victims of fake agile” – i.e. the people whose lives were ill-affected by the misapplication of agile. Is this similar to what you refer to as the “pseudo-master of Agile”? And in your opinion, what’s the biggest threat to an organization that adopts agile in a half-hearted manner?

[JK] There have been snake-oil salesmen since the dawn of mankind. If an organization is not able to hire good talent or good agile consultants, then a lot of damage can be done. Although, mostly, it would be the cost of lost opportunity going forward. That is, by not embracing and practicing “Real Agile ™,” the company wastes time, and time just might be money. However, as a counterpoint/cynical view… Most companies that do agile in name only are often large, matrix orgs, where the software is one small aspect of their business. Screwing up Walmart.com likely has very little impact versus someone screwing up their entire logistics system that moves products to stores.

6. We assume (and we could be wrong) that there was a healthy amount of debate amongst the authors of the Agile Manifesto. If so, what was the biggest point of contention within the group and how was it resolved?

[JK] I think the biggest area where we disagreed was the “how long is an iteration” – that is, how frequently should we expect tangible results? Many were at the two-week level, and others (Alistair) were at the 4+ weeks.

7. As a longtime agile coach, you’ve helped countless organization achieve better results with the approach. We’re curious to know when it hasn’t worked so well. Have you ever advised a company or organization to forgo agile in favor of another method? If so, what were the circumstances?

[JK] Agile is ALWAYS the right answer. What can be wrong about doing better with a set of resources? What can be wrong with reducing the gap in time between taking some action, and getting some feedback? Agile is a state of mind.

8. As with any manifesto, people are bound to misinterpret or misapply the main tenets. If you had to single out one particular way that agile has been misinterpreted, what would it be?

[JK] The classic missteps are usually NO documentation and NO design work upfront.

9. As we’re sure you are aware, Agile has its fair share of detractors and skeptics – and they can be a very vocal bunch. Why do you think agile is so strongly disliked in some quarters? And what is the one argument against agile that irks you the most?

[JK] I don’t really care to change people’s minds. The Agile Manifesto is irrefutable, as it gets to the root of human nature in a software development context – and is analogous to the founding documents of the United States. Agile promotes strong, disciplined individual and team responsibility and continuous participation from the development “citizenry.” It is much easier to fall back on some process just because someone wrote it in a book than it is to use your brain. No particular argument irks me, because I don’t care what people with closed minds think. A fool with a tool is still a fool.

10. Fill in the blank: The key to a successful agile testing team is: ____

[JK] being totally involved, working the upstream part of the process in addition to downstream verification.

11. Have you stayed in touch with the other authors of the Agile Manifesto? And have you considered “getting the gang back together” to publish any other materials?

[JK] I “hang” with Ron, Chet, Bob, Alistair, and Martin mostly in cyberspace. We got together for the 10th-year anniversary (the only Agile Alliance conference I went to, as it was paid for). We talked about getting together sooner than in another 10 years because we had a great time. But who knows if it will come to fruition. Much like the USA’s Founding Fathers, they got together for a momentous occasion but then went their separate ways.

12. What’s Jon Kern doing when he’s not helping companies improve their development process?

[JK] I like to mountain climb, hike, drive my Audi on the track or autocross, and ski. But when I am not doing that, I am writing ruby and rails code using MongoDB, git, and the wonderful world of ruby gems. I wish this remarkable constellation of language and tools existed when I was doing C++ way back when!

Read more on:

Testing the Limits With Jon Kern, Agile Manifesto Co-Author

 


Capybara Support for Multiple Submit Button Types

We had a case where we wanted to create smoke tests to ensure each customer’s subdomain was working (used pre- and post-deployment). Unfortunately, not all of the customer’s login pages were consistent in how to “submit” the login form 🙁

The Cucumber tests looked something like this:

  Scenario Outline: Check that the navigation is correctly customized per org
    Given I login as "<user>" of "<org>"
    And I am on the "<page>" page
    Then I should see only the expected "<menu_items>" in top nav bar
  Examples:
    | user       | org      | page | menu_items                                      |
    | acme-admin | acme     | Home | Home, Preferences, Feedback, Help, Logout       |
    | wyle-admin | wyle.com | Home | Home, Preferences, Links, Help, BI Tool, Logout |

Here is how I used Capybara’s find() method to get around this issue:

# A bit of a hack, org_name is normally a subdomain, but sometimes it is the complete domain
def login(user, org_name)
  # Use the below to automatically hit each user's org's server
  if org_name.include? '.com'
    Capybara.app_host = "http://#{org_name}"
  else
    Capybara.app_host = "http://#{org_name}.mydomain.com"
  end

  visit '/'
  fill_in 'username', :with => user
  fill_in 'userpwd', :with => '***'
  begin
    click_on 'submit'
  rescue Capybara::ElementNotFound
    page.find(:link_or_button, 'Log In')
    click_on 'Log In'
    rescue Capybara::ElementNotFound
      pending "Need to determine how to invoke the Login button for #{org_name} near Line ##{__LINE__} of #{__method__} in #{__FILE__} "
  end

  # Ensure that login was successful
  page.should_not have_content 'Login failed'
end

Since our analysts write a bunch of these tests, I also added in a helpful error message should a new org get added that has yet another style of login.


Keep your Cukes Narrow

It is not uncommon to wonder about how to create data for cukes…

  • Keep it high level?
    Given I have 4 products
  • Or explicitly define data; e.g., using a table

As a broad answer to this question, it all depends on what behavior you are specifying. There is no single, right way.

My usual rule of thumb is to limit the setup to just those parts of the problem domain “object graph” that you care to test in the current scenario. That helps draw attention to the most “narrow” bits of the system that are under test. Otherwise, if you continue to build everything from scratch for all scenarios, it is sometimes hard to see the purpose of the test because it is lost in all of the setup code. Sometimes you care about the details in the parent object, sometimes you want to test the sum of the parts, and other times you are focused only on a small aspect of the object model.

Examples

Suppose we are making bread, and we have a set of recipes. Each recipe has a set of ingredients. When we go to make 5 loaves of bread, the list of ingredients are properly re-sized and displayed so we can mix the dough and make a new batch of bread.

My first scenario might be detailed around setting up the core recipe/ingredient relationship:

Scenario: View a recipe
  Given I have the following recipe:
  |Name | Artisan Bread       |
  |Yield| 2 1-lb oblong loaves|
  |Prep | 30 min              |
  |Proof| 3 hours             |
  |Baking| 35 min. at 450 deg F|
  With the following ingredients:
  |Ingredient       |Quantity|
  |White Bread Flour|6 cups  |
  |Whole Wheat Flour|0.5 cups|
  |Granulated Yeast |1.5 Tbsp|
  |Coarse (sea) salt|1.5 Tbsp|
  |Water, 100 deg   |3 cups  |
  When I view the recipe
  Then I should see the recipe

So the above scenario gets us focused on the basic bits of the Recipe ----*-> Ingredients part of the model, and the desired behavior of what a recipe display looks like. Note: on the “Then” I chose not to focus on how it is displayed. I could have said something like “…with the recipe info at the top, followed by a list of ingredients.”

Now suppose the next scenario we tackled was the ability to “resize” the recipe to make some multiple of the base quantity? Do we need to repeat the same sort of setup as we did above? Or can we get away with defining less information? That is, using something like FactoryGirl, we could predefine some basic recipe data, if nothing specific is needed.

Scenario: Resize the batch to triple the yield
  Given I have a "Hard Tack" recipe that yields "25 cakes"
  With the following ingredients:
  |Ingredient | Quantity  |
  |Flour      | 4 cups    |
  |Water      | 2 cups    |
  |Salt       | 4 tsp     |
  When I resize the recipe to 3 times the size
  Then I should see the following ingredients
  |Ingredient | Quantity  |
  |Flour      | 12 cups   |
  |Water      |  6 cups   |
  |Salt       | 12 tsp    |
  And a yield of "75 cakes"

While the difference is subtle, the point to the above scenario is to not clutter it with the core recipe elements as were visible in the first scenario.

For a more extreme example of “narrowing” the focus, lets look at another scenario. This time, the business tells us they want users to be able to rate the recipes.

Scenario: Users can 'Like' a recipe
  Given I have a "Swedish Hard Tack" recipe
  When I Like the recipe
  Then I should see the Like Count increment by one
  And I should not be able to Like the recipe a second time

So here you can see there is no reason whatsoever to care about the details of the recipe. In fact, I could have left off the recipe name, but sometimes I like to keep some sense of the domain visible. Had I written the Given to show all of the detailed data, it would have obscured the meaning of the scenario, as this scenario would then look like the other scenarios.

Scenario: (BULKY VERSION!) Users can 'Like' a recipe
  Given I have the following recipe:
  |Name | Artisan Bread       |
  |Yield| 2 1-lb oblong loaves|
  |Prep | 30 min              |
  |Proof| 3 hours             |
  |Baking| 35 min. at 450 deg F|
  With the following ingredients:
  |Ingredient       |Quantity|
  |White Bread Flour|6 cups  |
  |Whole Wheat Flour|0.5 cups|
  |Granulated Yeast |1.5 Tbsp|
  |Coarse (sea) salt|1.5 Tbsp|
  |Water, 100 deg   |3 cups  |
  When I Like the recipe
  Then I should see the Like Count increment by one
  And I should not be able to Like the recipe a second time

Again, this is a subtle point. But making it easier for readers of the feature file to see differences without having to “strain” can improve understanding and reduce potential for errors.

For more on Cucumber, peruse my other posts on the topic, and see this “Crib Sheet.

Manual Cucumber Tests?

there was some discussion over on the cucumber list about manual testing.

cucumber is great at BDD, but it doesn’t mean it is the only test technique (preaching to choir) we should use.

i have learned it is critical to understand where automated tests shine, and where human testing is critical — and to not confuse the two.

as far as cuking manual tests, keeping the tests in one place seems like a good advantage (as described in Tim Walker’s cucum-bumbler wiki <g>).

the cucumber “ask” method looks interesting. maybe your testers could use the output to the console as-is, or (re-)write your own method to store the results somewhere else/output them differently.

From the cucumber code (cucumber-1.1.4/lib/cucumber/runtime/user_interface.rb):

# Suspends execution and prompts +question+ to the console (STDOUT).
# An operator (manual tester) can then enter a line of text and hit
# <ENTER>. The entered text is returned, and both +question+ and
# the result is added to the output using #puts.
# ...
def ask(question, timeout_seconds)
...

Sample Feature:

    ...
Scenario: View Users Listing
  Given I login as "Admin"
  When I view the list of users
  Then I should check the aesthetics

Step definition:

Then /^I should check the aesthetics$/ do
  ask("#{7.chr}Does the UI have that awesome look? [Yes/No]", 10).chomp.should =~ /yes/i
end

The output to the console looks like this:

Thanks for the pointer, Matt!

[notice]NOTE: it doesn’t play well with running guard/spork.[/notice]
The question pops up over in the guard terminal 🙁

Of course, if you are running a suite of manual tests, you probably don’t need to worry about the Rails stack being sluggish :-p

    Spork server for RSpec, Cucumber successfully started
    Running tests with args ["features/user.feature", "--tags", "@wip:3", "--wip", "--no-profile"]...
    Does the UI have that awesome look? [Yes/No]
    Yes
    ERROR: Unknown command Yes
    Done.

You Don’t Always Have to Follow “The” Rules

A user was asking the following on the Agile Modeling list:

What experience does anyone have about standards for stories written for
non-UI components? I’m working with a proxy PO who feels the standard story
format (As a <user> I want to <activity> so that <purpose>) simply won’t
work for something that doesn’t have a user interface. Imagine, for this
example, a project that has the sole purpose of encrypting data without any
user interaction.

For my needs, I often apply the principles behind the concepts, if not always the exact template of a suggested practice. Take for example, the use of my favorite tool, Cucumber, to write Acceptance Tests. Typically, cukes are written from the user point of view in classic “Given – When – Then.” But sometimes I like to use the cucumber style for testing APIs that I am building.

Here is one example at the Cucumber level (with the companion RSpec shown below):

Feature: Version 2.0: As we parse PDFs, we need to be able to collect a list of fonts
  as a way to help discern the structure of the parsed document based on
  heading levels, density of upper case text, and what not.

Scenario: Parsing a simple document
  Given a sample set of text
  |reference|points|value|
  |R9| 10| Patient: FRANKLIN, BENJAMIN|
  |R9| 10| CHIEF COMPLAINT:|
  When i parse the text
  And provide a set of base fonts
  |ref |basefont|
  | R9 | Helvetica-Bold|
  |R10 | Helvetica     |
  Then I should have the following font stats
  |reference|points|upper_cnt|lower_cnt|percent|
  |R9       | 10   | 4       | 1       | 83    |
  And the following font names
  |reference |points|basefont|
  |   R9     |  10  |Helvetica-Bold|
  |  R10     |  10  |Helvetica     |

Scenario Outline: Parsing a simple document
  When collecting a set of text , ,
  Then I should have  and  word counts and  Uppercase stats

  # Note: the counts are cumulative
  Examples:
    |reference|points|value|upper_cnt|lower_cnt|percent|
    | R9| 10| "Patient: FRANKLIN, BENJAMIN" |  2 |  1 | 66 |
    | R9| 10| "CHIEF COMPLAINT:"            |  4 |  1 | 83 |
    | R9| 10| "PRESCRIPTIONS"               |  5 |  1 | 89 |
    |R10|  9| "Motrin 600mg, Thirty (30), Take one q.i.d. as needed for pain, Note: Take with food, Refills: None."| 0 | 13 | 0 |
    |R10|  9| "(Discount Medication) < Michael L. Panera, PA-C 7/13/2010 17:40>"| 0 | 17 | 0 |

And here’s another one:

Feature: Extract meaningful data from Discharge Message

  Scenario: Extract headings
    Given a discharge message
    When the message is parsed
    Then I should see meaningful information, structured as headings and paragraphs
    And I can get formatted values for HTML display

  Scenario: Extract headings from second message
    Given a second discharge message
    When the message is parsed
    Then I can get formatted values for HTML display

But wait! There is more!

At the “Unit Test” level, RSpec’s can be made rather “english friendly” yet still be all about the underlying API as this diagram and snippet show:

RSpec for an API

The Font Collector RSpec tests

describe PdfParser do
  describe PdfParser::FontCollector do

    before(:all) do
      PdfParser::FontCollector.clear_all()
      ...
    end

    context "initialize" do
      it "should reject missing font reference" do
        ...
      end
      it "should reject missing points" do
        ...
      end
      it "should reject missing value" do
        ...
      end
      it "should accept valid inputs" do
        ...
      end

      it "should start off with simple stats" do
        ...
      end

      it "should recognize reference+size is unique" do
        ...
      end
    end

    context "clear" do
      it "should clear the font list" do
        ...
      end
    end

 

Factory Girl and MongoMapper

You were probably hoping for some Rosey the Riveter poster…

Factory Folder

Factory Folder

Instead, I am going to extend my small MongoMapper example to include Factory Girl. The steps are pretty simple:

  1. Go here to install…
  2. Create your factories
  3. Use the factories in Cucumber/RSpec

Factory Construction

I created a new “factories” folder under the spec folder:

The factories for User and Event are quite simple:

Factory.define :user do |u|
  u.name ('a'..'z').to_a.shuffle[0..7].join.capitalize
end

and

require 'factory_girl'
def dummy_word(len=6)
  ('a'..'z').to_a.shuffle[0..len].join.capitalize
end

def dummy_date
  secs_in_day = 24*60*60
  Time.now + (rand(60)*secs_in_day - 30)
end

Factory.define :event do |e|
  e.title "#{dummy_word} #{dummy_word 3} #{dummy_word 10}"
  e.date  dummy_date
end

Refactor Original Setup

Instead of using this style of test data creation:

@event = Event.create(:title => "Code Retreat Timbuktoo", :user => @fred)

We will use the new factory as follows:

@event = Factory(:event, :title => "Code Retreat Timbuktoo", :user => @fred)

Refactor Cucumber

The given went from this:

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => fred)
  end
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => harry)
  end
  Event.count.should == 20
end

to this – including refactoring out dummy_title, and reducing it to one loop:

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    evt = Factory(:event, :title => dummy_title,
                          :date  => dummy_date,
                          :user  => fred)
    evt = Factory(:event, :title => dummy_title,
                          :date  => dummy_date,
                          :user  => harry)
  end
  Event.count.should == 20
end

Subtle Details

The beauty of having tests is that I could easily mess around with getting some of the Factory Girl configuration stuff in the right place. Try something, run the test, adjust as needed until all are back to green.

The file features/support/env.rb got some additions so that Cucumber could find the factories:

$LOAD_PATH << File.expand_path('../../../app/model' , __FILE__)
require 'user'
require 'event'
require 'spec/factories/events.rb'
require 'spec/factories/users.rb'
load 'config/mongo_db.rb'

All the tests still pass!

More Complicated Example

For a project I work on, my factories look like this, with auto-creation of random IDs:

def random_months(months)
  day_in_secs = (24*60*60)
  (1+rand(months))*30*day_in_secs
end

# ----------- GROUP -----------
Factory.sequence :group_num do |n|
  "99#{n}#{rand(n)}"
end

Factory.define :group do |g|
    g.group_num {Factory.next(:group_num)}
    g.name "Greatest Group"
end
# ----------- ACCOUNT -----------
Factory.sequence :doctor_num do |n|
  "999992#{n}#{rand(200+n)}"
end

Factory.sequence :login do |n|
  "AB#{rand(n*68)}bx#{rand(200+n)}"
end

Factory.sequence :msid do |n|
  "CQ987Z12#{n}#{rand(n)}"
end

Factory.define :account do |a|
  pw = 'password'
  a.msid { Factory.next(:msid) }
  a.doctor_num { Factory.next(:doctor_num) }
  a.first_name "James"
  a.last_name "Jones"
  a.role 'user'
  a.password pw
  a.password_confirmation pw
  a.email Setting.get("AutoEmail")
  a.login { Factory.next(:login) }
end

# ----------- PATIENT -----------
Factory.sequence :patient_num do |n|
  "#{n}#{rand(300+n)}"
end

Factory.define :patient do |pt|
#  pt.patient_num "10000009"
  pt.patient_num {Factory.next(:patient_num)}
  pt.emr_num "1853286"
  pt.first_name "John"
  pt.last_name "Johnson"
  pt.dob {(Time.now - random_months(36))}
  pt.count_public_encounters 1
  pt.count_public_events 2
end

 

Cucumber, RSpec, MongoMapper, Git, Oh My!

Nick left a question on my many-to-many associations post. He wanted to know more about sorting by date and querying…

So I decided to (over achieve and) show how I would approach that as if I were adding a new feature:

  • Adding a “date” key field and an index
  • Using Cucumber to drive the new feature from the desired behavior à la BDD.
  • Querying with a date sort tacked on…

You can follow my progress from the earlier version to this one by examining the commit history:

Commit History

Commit History for Adding Dates to the Event Class

Looking at the “added initial event date functionality” commit, you can see how I added some new files to allow for Cucumber and MongoMapper:

Commits for Event Dates

Commits for Event Dates

For Cucumber, I added the “/features” stuff.

And since I wanted to start testing the code, I had to make the database functional, so I added “mongo_db.rb” – and I assume you have MongoDB installed and running locally.

##### MONGODB SETTINGS #####
MongoMapper.connection = Mongo::Connection.new('localhost', 27017, :pool_size => 5)
MongoMapper.database = "event-development"

I am never quite sure if there is a “perfect” way to wire up small, non-Rails apps like this one to use MongoDB. But what I have done works good enough to allow for a simple example to run.

BDD Cycle

So I began the BDD cycle by creating a “feature branch” and switching to a new branch from the current master branch:

git checkout -b event_dates master

Next I wrote the feature for the new behavior:

Scenario: Sort Events by Date
  Given A set of events
  When I display the events
  Then I should see them sorted by latest date first

When you run Cucumber, you will get the default code for steps – all pending of course.  Naturally, I did one step at a time, to take each one from “pending” to green. Working on the Given, then the When, and finally the Then, I came up with these steps:

Given

Here I wanted to generate a set of data so that we could see if the list was sorted properly. You can check out the code on github for the randomness baked into the “dummy_*” helper methods. And I wanted to create the events for two users.

Given /^A set of events$/ do
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => fred)
  end
  harry = User.find_or_create_by_name("harry")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => dummy_date,
                 :user => harry)
  end
  Event.count.should == 20
end

When

Sort of playing along as if this is a web request, I coded this step to return a “response” that is generated by the “list all events” class method. In true BDD fashion, this is the code I wish I had 🙂

When /^I display the events$/ do
  @response = Event.list_all
end

Then

The proof is in scanning the resultant “response” object to ensure date order is correct:

Then /^I should see them sorted by latest date first$/ do
  first = Date.parse(@response.third[0..10])
  last = Date.parse(@response.last[0..10])
  first.should > last
end

Outside – In

As soon as I ran the “When” I got a failure due to Event.list_all not existing. So off to the RSpec-land we go, to write the expectations for list_all. This is known as the “Outside-In” approach. (I learned this term from the excellent RSpec book, and it looks like you can watch a video about it here.)

The behavior expressed above (in Cucumber) can be thought of as more of an “outer,” acceptance/integration style of test. Typically it would be the User Interface (UI) – but I have been known to blur that line, since not all code is about UI and since Cucumber is so darn fun to use. Working at this outer level often leads to expressing what is expected of our actual code; in this case, that Event have a class-level method that returns a list of it’s instances (a.k.a., documents). Since we are talking about the behavior of a class, that is more of the “inside” of the application. Not something that an external user might care so much about directly, but rather something that supports the end behavior in an indirect fashion. For the “inside” we turn to RSpec (basically a better-than-unit-test, unit test tool).

  • Outside ≈ Feature ≈ Cucumber
  • Inside ≈ unit test ≈ RSpec
  describe "#list_all" do
    it "should show each event, ordered by date" do
      response = Event.list_all
      response.should_not be_empty
      response.class.should == Array
      response.size.should == Event.count + 2 #for title and column header
      # Yes, you should not output stuff as part of your tests, but this *is* our UI :-)
      puts response
    end
  end

Many times, my initial pass at a new method is to simply return what is expected. Then write another test to make that fail. Sort of “sneak up on the answer.” But here it was easy enough to simply output some real text from the get-go:

  def self.list_all
    response = []
    response << "%s %s %s" % ["*"*10, "LIST OF EVENTS", "*"*10]
    response << "%6s %15s               %s" % ["Date", "TITLE", "Attendees/Interested/Likes"]
    events = Event.all(:order => 'date desc')
    events.each {|e| response << e.to_summary}
    response
  end

Oops. LOL (:-D) While writing this post, I found a mistake when testing my code a bit further than my initial commit.

I decided to remove the order part of the query, which revealed that the Cucumber feature still passed. Crap! So, it wasn’t so easy after all! Dope.

    events = Event.all

Second Attempt

My initial way of generating records resulted in the documents magically being in the right date order by default. Tests passed, but the test was wrong – not vigorous enough testing!

Note to self:
no matter how trivial things seem, write failing tests
that contradict each other – so to speak.

So, I tweaked the document generator to better randomize the list of events such that we won’t accidentally have them all in proper order by default:

  ...
  fred = User.find_or_create_by_name("fred")
  (1..10).each do
    Event.create(:title=>"#{dummy_word} #{dummy_word 3} #{dummy_word 10}",
                 :date => Time.now + (rand(60)*secs_in_day - 30),
                 :user => fred)
  end
  ...

And, instead of just spot-checking the order, here is a new RSpec test to ensure each event is in proper order, date-wise:

    it "should show each event, ordered by date" do
      response = Event.list_all
      response.should_not be_empty
      response.class.should == Array
      response.size.should == Event.count + 2 #for title and column header
      r_prior_date = Date.parse(response[2][0..10])
      response[3..response.size].each do |r|
        date = Date.parse(r[0..10])
        date.should < r_prior_date
        r_prior_date = date
      end
      # Yes, you should not output stuff as part of your tests, but this *is* our UI :-)
      puts response
    end

Now we’re talking! A failed test:

'Event#list_all should show each event, ordered by date' FAILED
expected: < Wed, 20 Apr 2011,
     got:   Wed, 20 Apr 2011

And similarly, I re-wrote the Cucumber test. Funny thing, further testing revealed that the error above was not actually a legitimate fail as it turns out! I discovered I needed “<=” instead of just “<“– sometimes the simplest things aren’t so simple after all. Especially when it comes to setting up sample data.

Then /^I should see them sorted by latest date first$/ do
  last_date = Date.parse(@response.third[0..10])
  @response[3..@response.size].each do |r|
    date = Date.parse(r[0..10])
    date.should <= last_date
    last_date = date
  end
end

And I got the above test to fail by “stepping back” and removing the “order by” clause to get me back to an original, non-sorted listing. Good! Now we can step forward again and try to get the functionality that we are looking for to work.

Cucumber Failing Tests

Cucumber Failing Tests

I re-enabled the order clause to see if the tests would now pass:

events = Event.all(:order => 'date desc')

And, fortunately, the tests are indeed passing:

Cucumber Passing Tests

Cucumber Passing Tests

Commit on Green

Once you get the bits of functionality working, commit (even if you still have pendings). Committing locally has no downside 🙂 Here I will commit and push to the repo (the “$” is my prompt (well, not really), and the #comments are not part of the command line!):

$git status #You can see your changes
$git commit -a -m "added initial event date functionality"  #commit your changes
$git checkout master #switch to the master branch
$git merge --no-ff event_dates #merge all of your local feature branch commits, preserving each
$git push origin master #Pump it up to the repo
$git branch -d event_dates #Get rid of the feature branch

This rhythm gets to be very familiar.

More on Querying

You may have noticed some of the queries above, and this was one of Nick’s questions…

With MongoMapper, you can chain Plucky queries as follows:

  def self.list_all(a_user=nil)
    response = []
    response << "%s %s %s" % ["*"*10, "LIST OF EVENTS", "*"*10]
    response << "%6s %15s               %s" % ["Date", "TITLE", "Attendees/Interested/Likes"]      events = nil     if a_user.nil?       events = Event.all(:order => 'date desc')
    else
      events = Event.where(:user => a_user).all.sort(:date.desc)
    end
    events.each {|e| response << e.to_summary}
    response
  end

I added a new feature that shows off the above query, quickly ran through the entire process again, from git checkout to git push, with Cucumber and RSpec and code in between. You can find it all in the source code.

Cucumber Show Events For User

Cucumber Show Events For User

Git is really an amazing revision control tool… I can’t imagine using anything else now. Here is an example of looking at the “Network Graph” of my little project:

github network graph

Github Network Graph

Refactoring Cucumber Features

I was inspired by this post (You’re Cuking It Wrong) to re-look at some feature specs to see if I could improve them.

I went from here:

Feature: Season Management
...
Scenario: Create a Season
  Given an organization like "My YMCA"
  When I add a season like "Summer Basketball" for "My YMCA" from "1 May" to "31 August"
  Then I should see the "Summer Basketball" season for "My YMCA" in a list of the organization's seasons

Scenario: Add a Registration Setup
  Given the "Summer Basketball" season for "My YMCA"
  When I add a setup like "Summer Signup" open from "1 April" to "30 April"
  Then I should see "Summer Signup" under the "Summer Basketball" season

to here:

Scenario: Create a Season
  Given an organization like "My YMCA"
  When I add "My Season" season
  Then I should see the "My Season" season

Scenario: Add a Registration Setup
  Given the "My Season" season for "My YMCA"
  When I add a setup like "My Season Signup"
  Then I should be able to signup under "My Season"

This also led to improvements in the steps. For example, by getting rid of the unnecessary dates, I also removed some unintended “brittleness.” As an example, this:

When /^I add a season like "([^"]*)" for "([^"]*)" from "([^"]*)" to "([^"]*)"$/ do |season, org, from, to|
  visit organizations_path
  click_link org
  click_link "New Season"
  fill_in "Name" , :with => season
  fill_in "season_starts_at" , :with => from
  fill_in "season_ends_at" , :with => to
  click_button "Create"
end

to:

When /^I add "([^"]*)" season$/ do |season|
  visit organizations_path
  click_link @org.name # Created in a prior scenario, no need to pass in
  from = Time.now      # Create more appropriate from/to values
  to = from + 120*24.hours
  click_link "New Season"
  fill_in "Name" , :with => season
  fill_in "season_starts_at" , :with => from
  fill_in "season_ends_at" , :with => to
  click_button "Create"
end

Subtle, maybe, but certainly more clear and less likely to need attention if/when the solution changes.