Build Your Own Real Estate Listing Service with Ruby on Rails and Sphinx Part 1

In this tutorial we will use Ruby on Rails with the Sphinx search engine to build a small real estate listing service. We will use Sphinx to provide a google like free text search with results ordered by relevance.

This tutorial is splitted in two parts. In this first part it covers the preparation of the development stack and the implementation of the first feature with behavior driven development.

In the second part we will implement the second feature, the search with sphinx.

Let’s begin with creating the site and installing all needed gems. If you already know how to set up a complete RSpec, Capybara, Guard, MySQL, Sphinx environment you can skip the following paragraphs.

Setup for Behavior Driven Development

We create a new rails application with the follwing command:

1
  rails new tutorial -T

The -T option prevents that the rails generator creates the standard test directory. Next we add all needed gems to Gemfile. Like in our last tutorials we use Twitter Bootstrap to make the output a little bit nicer:

1
2
3
4
5
6
7
group :assets do
  ...
  gem 'less-rails-bootstrap'
  gem 'therubyracer'
  gem 'twitter-bootstrap-rails'
  ...
end

For behavior driven development we add rspec-rails and factory_girl_rails. Please take care that these gems have to be inside the development and test group.

1
2
3
4
group :development, :test do
  gem 'rspec-rails', '~> 2.0'
  gem 'factory_girl_rails', '~> 4.0'
end

For integration testing we use capybara together with guard. This time we need them only inside the test group.

1
2
3
4
5
6
group :test do
  gem 'capybara', '~>2.0.2'
  gem 'guard'
  gem 'guard-rspec'
  gem 'growl'
end

We will use guard and guard-rspec for automatic testing. And last but not least we add the components for Sphinx.

Installing Sphinx

Before we add the gems we first install the sphinx package. On mac systems with port installed we can use following command:

1
  sudo port install sphinx

Packages for other systems we can find here:

Sphinx Download Site

If we have installed the package properly we can add to our Gemfile:

1
2
  gem 'mysql2', '0.3.12b4'
  gem 'thinking-sphinx', '3.0.0'

In this tutorial we use the mysql2 gem working together with Sphinx. As far as I know Sphinx doesn’t support SQLite, but it’s possible to combine it with Postgresql. Now we install all gems with

1
  bundle install

Preparing Mysql

Because we use in this tutorial the mysql database system we have to change our database.yml like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
development:
  adapter: mysql2
  database: tutorial_development
  username: <your_db_username>
  password: <your pw>
  encoding: utf8
  host: 127.0.0.1
  port: 3306

test:
  adapter: mysql2
  database: tutorial_test
  username: <your_db_username>
  password: <your pw>
  encoding: utf8
  host: 127.0.0.1
  port: 3306
...

Now we create all databases with:

1
  bundle exec rake db:create:all

Setting up RSpec and Guard

With the command

1
  bundle exec rails generate rspec:install

we create a basic structure for our test suite. To make Rspec working together with Capybara we open the spec/spec_helper.rb file and add:

1
  require 'capybara/rspec'

Next we open the config/application.rb file and configure the test framework generator like this:

1
2
3
4
5
6
7
8
9
10
module Tutorial
  ...
  class Application < Rails::Application
    config.generators do |g|
      g.test_framework :rspec, fixtures: true, view_specs: false, helper_specs: false,
      routing_specs: false, controller_specs: true, request_specs: false
      g.fixtures_replacement :factory_girl, dir: "spec/factories"
    end
  ...
end

The lines above define the components of the testing framework the Rails generators creates automatically and we declare that we want factory_girl as fixtures replacement. Because we want to do our integration test with capybara we turn of the generation of request specs.

We can’t go into much detail here. If you like to get more information about RSpec/Capybara testing I recommend the book of

Aaron Summer: Testing with RSpec.

Next we take care about autotesting. In this tutorial we use guard to do this job. To generate a Guardfile for configuration we use this command:

1
  guard init rspec

The generated Guardfile is located in our main directory. It contains all importent settings for watching RSpec and Capybara. Because we don’t use route specs we remove this line from our Guardfile:

1
  watch('config/routes.rb') { "spec/routing" }

Now we can observe of our test suite with:

1
  guard

automatically. After all preparation is done we can start with the development.

Basic Features for a Real Estate Listing Service

In this tutorial we start with two simple basic features for our small real estate listing service. The following lines describe the basic behavior of the system:

  • As a user of this service
  • I want to visit the homepage and see the latest 5 real estate listing entries
  • so that I have fast access to the latest offerings.
  • As a user of this service
  • I want to enter a search text and get the relevant search results
  • so that I can find the right property

With this two features in mind we can write our integration tests.

Integration Tests with Capybara

At first we add the ‘features’ directory inside /spec. Inside spec/features/ we create a file latest_entries_spec.rb. In this file we add following lines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
require 'spec_helper'

feature 'Latest 5 properties on homepage', %q{
  As a user of this service
  I want to visit the homepage and see the latest 5 real estate listing entries
  so that I have fast access to the latest offerings.
} do

  scenario 'Show the latest 5 properties on startpage' do
    visit root_path
    expect(page).to have_content 'Property 6'
    expect(page).to have_content 'Property 5'
    expect(page).to have_content 'Property 4'
    expect(page).to have_content 'Property 3'
    expect(page).to have_content 'Property 2'  
  end

end

We use the new capybara syntax for this integration test. The guard should execute latest_entries_spec.rb automatically. As expected this test fails because we don’t have any properties defined. We will change that by using:

1
  rails g model Property title:string description:text price:decimal city_name:string

This rails generator creates a property model with some attributes. For now we only use four attributes, but you can add more if you like. Additionally the generator creates the spec files for our test framework. Next we use the commands:

1
2
  bundle exec rake db:migrate
  bundle exec rake db:test:clone

to populate our databases. Now we have a property model and we continue to work on our test suite.

Fixtures with Factory_Girl

Factory_Girl is a gem that allows us to work with data fixtures more comfortably. The rails generator already created the properties.rb file inside the spec/factories/ directory. We modify this file so that it looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  FactoryGirl.define do
    factory :property do
      sequence(:title) { |i| "Apartment #{i}"}
      price "2000000"
      description "Apartment description"
      city_name "Defaut City Name"

      trait :madison do
        description "Great butler service. Doorman. Elevator. Gym. Roofdeck. Pets OK, Steps to Madison Square Park, Union Square, and great shopping and restaurants."
      end

      trait :five_star_hotel do
        description "Fabulous bright and sunny Cipriani Penthouse. Butler service. Five Star Hotel Service in one of the most luxurious buildings in Financial District."
      end

      trait :ny do
        city_name "New York"
      end

      trait :la do
        city_name "Los Angeles"
      end
    end
  end

We use the trait declaration to make this factory definition more flexible in usage. Traits are like shortcuts that allows us to create fixtures with different descriptions and city_names easily.

It’s not necessary for our first feature but in the second part of the tutorial it will simplify the free search testing.

To make the usage of factories more comfortable we add to spec/spec_helper.rb follwing line:

1
  config.include FactoryGirl::Syntax::Methods

This allowes us to use a shorter syntax in our spec files. We can shortcut FactoryGirl.create(….) with create(….).

After we created our factory definition of property we can continue with our behavior tests. For this we open our property_spec.rb in the spec/models directory.

Why not continue with our integration test? After we created the model, we want to make sure that this model works like intented. To ensure of this we write a model spec that fails and write the code that makes this spec pass.

For each feature we will use following steps for behavior driven development in this tutorial:

  1. 1. Write a integration test in the directory spec/features. This test fails.
  2. 2. Create a component(model,controller,view) in order to make the integration test pass.
  3. 3. Create the component spec to ensure that this part work like intented. This spec fails.
  4. 4. Create the code of the component until the component spec passes.
  5. 5. After the spec passes refactor the code of the component.
  6. 6. Repeat steps 2 to 5 until the integration test passes.
  7. 7. If the integration test passes write the next integration test an start again.

Let’s open the property_spec.rb in the directory spec/models/. We want the behavior that all attributes of the property model are required at creation. We don’t want that one of them left blank. So we modify our property_spec.rb that it looks like follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
  require 'spec_helper'

  describe Property do
    it "is valid with title, description, cityname and price" do
      p = create(:property, :ny, :madison)
      expect(p).to be_valid
    end

    it "is invalid without title" do
      p = build(:property, :ny, :madison, :title => nil)
      expect(p).to have(1).errors_on(:title)
    end

    it "is invalid without description" do
      p = build(:property, :ny, :description => nil)
      expect(p).to have(1).errors_on(:description)
    end

    it "is invalid without cityname" do
      p = build(:property, :madison, :city_name => nil)
      expect(p).to have(1).errors_on(:city_name)
    end

    it "is invalid without a price" do
      p = build(:property, :madison, :price => nil)
      expect(p).to have(1).errors_on(:price)
    end
  end

This spec will fail because there is no validation code in the model. To pass this test we add to our property model:

1
2
3
4
  validates :title, :presence => true
  validates :description, :presence => true
  validates :city_name, :presence => true
  validates :price, :presence => true

Now the model test should pass and we can work with our new property model.

In our integration test we add to latest_entries_spec.rb a new background block that it looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
feature 'Latest 5 properties on homepage', %q{
  As a user of this service
  I want to visit the homepage and see the latest 5 real estate listing entries
  so that I have fast access to the latest offerings.
} do

  background do
    p1 = create(:property,:title => 'Property 1', :created_at => 6.days.ago)
    p2 = create(:property,:title => 'Property 2', :created_at => 5.days.ago)
    p3 = create(:property,:title => 'Property 3', :created_at => 4.days.ago)
    p4 = create(:property,:title => 'Property 4', :created_at => 3.days.ago)
    p5 = create(:property,:title => 'Property 5', :created_at => 2.days.ago)
    p6 = create(:property,:title => 'Property 6', :created_at => 1.day.ago)
  end

  ...

These lines create six properties with different titles. We need six properties to test that only the latest five properties will displayed. That is also the reason that we specify the created_at attribute.

After saving our spec file we notice that the integration test still fails. That’s ok. Even if we create these properties there is no output on our website. So we go back to step 2 and add one more component to make our integration test pass. This time we add a controller with:

1
  rails g controller home index

The define the controller behavior we modify our home_controller_spec.rb like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
  require 'spec_helper'

  describe HomeController do

    describe "GET 'index'" do
      it "delivers the latest 5 listings" do
        p1 = create(:property,:created_at => 6.days.ago)
        p2 = create(:property,:created_at => 5.days.ago)
        p3 = create(:property,:created_at => 4.days.ago)
        p4 = create(:property,:created_at => 3.days.ago)
        p5 = create(:property,:created_at => 2.days.ago)
        p6 = create(:property,:created_at => 1.day.ago)
        get 'index'
        expect(assigns(:properties)).to match_array [p2,p3,p4,p5,p6]
      end
    end

  end

The behavior described in this spec is quite simple. We create six properties but only the latest five properties should be returned by the home#index method.

This spec fails because there is no code in the controller. We can make it pass if we change the code of the index method so that is looks like this:

1
2
3
4
5
6
7
  class HomeController < ApplicationController

    def index
      @properties = Property.order("created_at DESC").limit(5)
    end

  end

The controller test will pass now but the integration test still fails. We have now the latest five properties but they are not displayed on the frontpage. We have to change the index.html.erb in views/home/. After the change it should look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
  <div class="page-header">
    <h1>Latest properties</h1>
  </div>
 
  <table class="table table-striped">
    <thead>
      <tr>
        <th>Title</th>
        <th>Description</th>
        <th>Price</th>
        <th>City</th>
      </tr>
    </thead>
    <tbody>
      <% @properties.each do |p| %>
      <tr>
        <td>
          <%= p.title %>
        </td>
        <td>
          <%= p.description %>
        </td>
        <td>
          <%= p.price %>
        </td>
        <td>
          <%= p.city_name %>
        </td>
      </tr>
      <% end %>
    </tbody>
  </table>

Our integration should fail again because we have do define the root path in our config/routes.rb file. If we add

1
  root :to => 'home#index'

our first integration test should pass. We don’t add an view spec test here because the behavior is already covered by our integration test.

Summary

In this tutorial we started with behavior driven development of a small real estate listing service. We developed our first simple feature with the support of Rspec and Capybara tests.

As you have noticed the controller test is quite similar to the integration test. In this case we could omit the test because it doesn’t cover anything new. For learning purposes I left the test in the tutorial.

In Next Tutorial

In the next part of this tutorial we will implement the second feature, the free text search of properties covering integration tests of search requests with sphinx.

If you like this tutorial please share it on your social networks. If you have tips for improvement or any other information please leave a comment.

10 Responses to “Build Your Own Real Estate Listing Service with Ruby on Rails and Sphinx Part 1”

  1. Jeff

    I’m looking forward to the next tutorial. This is good stuff. Thank you.

    Reply
  2. Marek

    excellent tutorial and site. Easy to follow with a valuable content. Looking forward for next issues….

    Reply
  3. Rob

    I found a couple of small things missing from the tutorial:
    #1 – You need to include “require ‘factory_girl’” in the spec_helper.rb
    #2 – There is a typo for: “config.include FactoryGirl::SyntaxMethods” – it should be “config.include FactoryGirl::Syntax::Methods”
    #3 – It wasn’t obvious to me where the “background” keyword goes in the latest_entries_spec.rb. It belongs inside the “feature”
    #4 – To get the integration test to pass you have to remove the index.html file from the public directory
    #5 – After changing the routes.rb file, guard started complaining about a missing routes spec. I commented out the line in the Guardfile watching the routes.rb file

    Thx!

    rb

    Reply
    • Lars

      Hi Rob,

      thank you for your supporting comments. To the points:

      #1 – My test suite here works fine without an additional “require ‘factory_girl’” in the spec_helper.rb. Do you have any reference to documentation that this line is required?
      #2 – Fixed.
      #3 – Fixed.
      #4 – Sorry, it was obvious for me. But you are right.
      #5 – Fixed.

      Kind regards,
      Lars

      Reply
  4. Rob Bastian

    Lars –

    You’re right. I removed require ‘factory_girl’ from my spec_helper.rb and all the tests pass.

    My bad!

    rb

    Reply
  5. Matt

    Great tutorial Lars. This is the first rails tutorial I have worked through where the setup actually worked with ease. I am also amazed how well you covered the BDD principles so quickly. Thanks

    Reply

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>