Speed Up Integration Tests with before(:all) blocks

The other day I came across a great use-case for the RSpec before(:all) block: Integration tests.

I don't typically use before(:all) blocks in my RSspec tests. Generally, a before(:each) (or just before as shorthand) is what I want. When you find yourself modifying data in my individual specs, then a before(:all) block can have some unexpected side effects.

Whats the difference between before(:all) and before(:each)?

before(:each) is run before each individual spec test (the it '...' statement), where as the before(:all) block is only one once for the entire context. Lets run an RSpec test to get the point across:

describe 'before blocks' do

  before(:all) do
    puts "before(:all) block hit!"
  end

  before(:each) do
    puts "before(:each) block hit!"
  end

  it 'Hits the before(:all) block once' do
    puts "\tFirst assertion."
    expect(true).to be(true)
  end

  it 'Hits the before(:each) block twice' do
    puts "\tSecond assertion."
    expect(true).to be(true)
  end

end

# Running rspec will print something like this:

#=> before(:all) block hit!
#=> before(:each) block hit!
#=>   First assertion.
#=> .before(:each) block hit!
#=>   Second assertion.

When we run this test, you can see that the before(:all) block is run once for the entire 'Describe' block, but the before(:each) block is run before every 'it' assertion. Makes sense.

So, why can the before(:all) block get you into trouble? Well, let's say you are creating a record in your before block. You might think it a good idea to save time by reducing the amount of db create calls, but if you are modifying that record within a single spec, that change will propagate to any downstream spec test. Let's see another example.

describe 'before(:all) downsides' do

  before(:all) do
    @valid_user = User.new(name: "John Smith", email: 'myemail@email.com')
  end

  it 'makes some long lasting changes' do
    @valid_user.email = "invalid_email"
    expect(@valid_user.valid?).to eq(false)
  end

  it 'that affect other tests' do
    @valid_user.update_user_with_valid_data(name: "John Smithson")
    # This test will fail
    expect(@valid_user.valid?).to eq(true)
  end

end

If you are not aware of the before(:all) characteristics, then you can find yourself troubleshooting a downstream failing test, when the issue is actually do to "bleeding" tests. Most of the time, I find before(:each) blocks cleaner. You start off with a nice clean slate for every test, without duplicating your code.

So when is it a good idea to use before(:all) blocks? Well one case that I ran into the other day, is full integration tests for my Sinatra API. This Sinatra API responds JSON formatted data. Lots of data with lots of fields. Also my API supports certain query parameters to filter data, or include extra data. I originally write my rspec tests simliar to this:

describe 'My API' do

  context 'includeSource' do
    before do
      get '/data?includeSource=all'
      @full_response = JSON.parse(last_response.body)

      get '/data?includeSource=db1'
      @db1_only_response = JSON.parse(last_response.body)
    end

    it 'returns only db1 data when includeSource=db1' do
      expect(@db1_only_response["Data"].select { |d| d["source"] != db1}.count).to eq(0)
    end

    it 'returns all db1 records that are included with the full' do
      full_response_db1_count = @full_response["Data"].select { |d| d["source"] == db1}.count
      db1_only_count = @db1_only_response["Data"].select { |d| d["source"] == db1}.count

      expect(db1_only_count).to eq(full_response_db1_count)
    end

  end

end

Both tests succeeded, but my rspec tests make 4 get calls, 2 for each 'it' assertion. Since my assertions are not modifying data, but rather validating data, I can use the before(:all) block instead. By changing

    before do
      get '/data?includeSource=all'
      @full_response = JSON.parse(last_response.body)

      get '/data?includeSource=db1'
      @db1_only_response = JSON.parse(last_response.body)
    end

to

    before(:all) do
      get '/data?includeSource=all'
      @full_response = JSON.parse(last_response.body)

      get '/data?includeSource=db1'
      @db1_only_response = JSON.parse(last_response.body)
    end

I reduce the number of get calls by half. For a lot of cases, this might not save a lot of time, but for me it saved quite a bit of time. I ended up writing about 80 spec tests to validate full integration flows and was able to reduce the total time by 70%.