Build Your Own Real Estate Listing Service with Ruby on Rails and Sphinx: Part 2

Share It!

In part 1 of this tutorial we started to build a small real estate listing site and we got our first integration test to pass. In this tutorial we implement the free text search with sphinx.

If you haven’t read the first part of the tutorial you can read it here: Part 1.

Because we use in this tutorial behavior driven development again we start with writing an integation test. We create inside the spec/features directory a new file search_spec.rb. To this file we add following code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
  require 'spec_helper'

  feature 'Search for properties', %q{
    As a user of this service
    I want to enter a search text and get the relevant search results
    so that I can find the right property
  } do

    background do
      @p1 = create(:property,:ny,:madison,:title => "Property 1")
      @p2 = create(:property,:ny,:five_star_hotel,:title => "Property 2")
      @p3 = create(:property,:city_name => "Washington", :title => "Property 3")
    end

    scenario 'Search property with butler service' do
      visit root_path
      fill_in 'query', with: 'butler service'
      click_button 'Search'
      expect(current_path).to eq search_path
      expect(page).to have_content 'Property 1'
      expect(page).to have_content 'Property 2'
      expect(page).not_to have_content 'Property 3'
    end

    scenario 'Search property with five star hotel service' do
      visit root_path
      fill_in 'query', with: 'five star hotel service'
      click_button 'Search'
      expect(current_path).to eq search_path
      expect(page).to have_content 'Property 2'
      expect(page).not_to have_content 'Property 1'
      expect(page).not_to have_content 'Property 3'
    end

    scenario 'Search property near Madison Square Park' do
      visit root_path
      fill_in 'query', with: 'Madison Square Park'
      click_button 'Search'
      expect(current_path).to eq search_path
      expect(page).to have_content 'Property 1'
      expect(page).not_to have_content 'Property 2'
      expect(page).not_to have_content 'Property 3'
    end

  end

The logic behind this integration test is quite simple. In the background we create three fixtures with different descriptions and titles. We prepared this fixtures in our last tutorial. Then we create three search scenarios.

In the first scenario we search for a property with ‘butler service’. Property 1 and Property 2 have ‘butler service’ in their description texts. So we expect to get only these two results from the search.

In the second scenario we search for a property with ‘five start hotel service’. Only Property 2 has a ‘five star hotel service’ in it’s description. So we expect only Property 2 from the search.

The last scenario is a search for a property near Madison Square Park. Property 1 has the proper description and so we expect only Property 1 in the search results.

Property 3 doesn’t have any matching description texts to get found within our searches. So it should not appear in any of our three search results.

This test fails as expected. We don’t have any search form and we don’t have a search controller. We have only the property model. So we start with making a search form. For this we open our home/index.html.erb and add the following lines:

1
2
3
4
  <form class="form-search" action="/search" method="GET">
    <input type="text" name="query" class="input-medium search-query">
    <button type="submit" class="btn">Search</button>
  </form>

There is no need for a view spec here. This code lines are covered by our integration test. We re-run our integation test and it fails with another error message. There isn’t any search controller executing the search. So we add a new controller with the command:

1
  rails g controller searches show

I know there are a lot of discussions about how to integrate a search function within the REST paradigma. In this tutorial we use a ‘GET /search’ with a query parameter to make a search request.

Here we skip the controller spec too. The behavior of the controller is covered by our integation test. There is no need for double coverage.

After we created our controller have to update the routes.rb with this line:

1
  resource :search

This resource declaration enables the GET /search route for our search. The integration test still fails because there is no search functionality. We will change it by using the sphinx search engine.

At first we create a new directory app/indices and add the file property_index.rb. In this file we add following code:

1
2
3
4
  ThinkingSphinx::Index.define :property, :with => :active_record do
    indexes title
    indexes description
  end

This declaration tells the sphinx search engine that we want to index the title and description attributes. There is no further configuration for now here. Next we can add to our show method of the search controller following line:

1
  @properties = Property.search params[:query]

This line above simply add the full text search via sphinx.

Before we can go further we have to configure our test environment for testing with sphinx. For this we open the spec_helper and modify the code that it looks like follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
  RSpec.configure do |config|

    config.before(:suite) do
      ThinkingSphinx::Test.init
      ThinkingSphinx::Test.start_with_autostop
    end

    config.before(:each) do
      ThinkingSphinx::Test.index if example.metadata[:js]
    end

    config.after(:each) do
      Property.delete_all
    end

    config.include FactoryGirl::Syntax::Methods

    ...

    config.use_transactional_fixtures = false
   
    ...
   
  end

The most importent thing is that we have to set transactional_fixtures to false. The sphinx gem won’t work if it’s set to true. With the config.before(:suite) block we initialise the gem and restart the sphinx server. Because we have transactional fixtures set to false we have to clean up the test database by ourself. In the config.after(:each) block we remove all property entries from the database.

If the before(:suite) block throws an error then we have to check that the sphinx search engine isn’t still running in the background. The sphinx search engine can only run once. Sphinx runs as ‘searchd’ daemon in the background. We have to stop it with the command:

1
  bundle exec rake ts:stop

before we can run our test suite. After we made sure that no daemon is running we enter our search_spec.rb feature and modify the background block that it looks like this:

1
2
3
4
5
6
7
  background do
    @p1 = create(:property,:ny,:madison,:title => "Property 1")
    @p2 = create(:property,:ny,:five_star_hotel,:title => "Property 2")
    @p3 = create(:property,:city_name => "Washington", :title => "Property 3")
    ThinkingSphinx::Test.index
    sleep 0.25 until Dir[Rails.root.join('db', 'sphinx', 'test', '*.{new,tmp}.*')].empty?
  end

After we created the three properties we index them. Because we don’t now how long it takes til sphinx reindexed the test data we let the test sleeping until the indexing is finished.

If we run the integration test again we still get an failure. The reason is that there is no application code in the views/searches/show.html.erb file. We modify this file that it looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
  <div class="page-header">
    <h1>Search Results</h1>
  </div>

  <form class="form-search" action="/search" method="GET">
    <input type="text" name="query" class="input-medium search-query" value="<%= params[:query]%>">
    <button type="submit" class="btn">Search</button>
  </form>

  <table class="table table-striped">
    <thead>
      <tr>
        <th>Title</th>
        <th>Description</th>
        <th>Price</th>
        <th>City</th>
      </tr>
    </thead>
    <tbody>
      <% @properties.each do |p| %>
      <tr>
        <td>
          <%= p.title %>
        </td>
        <td>
          <%= p.description %>
        </td>
        <td>
          <%= p.price %>
        </td>
        <td>
          <%= p.city_name %>
        </td>
      </tr>
      <% end %>
    </tbody>
  </table>

If we save this file and re-run the integration test the test should pass and we should get an output like this:

1
2
3
4
5
6
7
8
  22:09:44 - INFO - Running: spec
  ...........

  Finished in 4.13 seconds
  11 examples, 0 failures


  Randomized with seed 44054

Summary

In this tutorial we implemented the second feature of our small real estate listing service using behavior driven development. Now we have two important basic functions for our site we can build on.

In the next tutorials we will add more features. This features will thematize user management with different roles, creating, updating and deleting of properties by authorized users, advanced search with sphinx, geocoding and mapping with google maps and many more.

If you like this tutorial please share it on your social networks. If you have tips for improvement or any other information please leave a comment.

Share It!

Comments

  1. Rob Bastian says

    In case anybody else runs across this problem, I’ll post it here.

    I was receiving this error on query submit: Can’t connect to MySQL server on ’127.0.0.1′ (61). As it turns out the Sphinx search daemon wasn’t running. The error message is misleading but the stack trace indicated it was coming from Sphinx.

    • says

      Hi Rob, thx for the contribution. This commands:

      1
      2
        bundle exec rake ts:start
        bundle exec rake ts:stop

      start and stop the Sphinx search daemon in the development environment. Please keep in mind to shut down the sphinx daemon if you return to the test suite.

      • Matt says

        I was encountering the issue:

        Can’t connect to MySQL server on ’127.0.0.1′ (61).

        I installed sphinx using brew instead of port. What fixed the problem for me was uninstalling sphinx then installing using:

        brew install sphinx –mysql

  2. Marek says

    Hi Lars,
    just wondering..on what kind of projects would you choose Sphinx search options over other search engines …e.g elastic search … what is the advantages of using sphinx according to your experience….?

    • says

      Hi Marek, using Sphinx is my personal preference. I have worked with it on a similar project some years ago. For a real estate listing service you need free text search in combination with ‘hard’ parameters. For example searching a apartment with white marble in a price range from 10Mio to 20Mio. Sphinx can do it and it can do it fast. I don’t want to claim that it is the perfect choice but Sphinx works well for this kind of services.

  3. Raghu says

    I am waiting for next steps. I think it has been very long since part -2 has been posted.
    Sorry to say this. All that was explained is basic. It didn’t yet involve thinking spinx.
    I am eagerly waiting for that.

    Thanks,
    Raghu

    • says

      Hi Raghu,
      you are right, 10 days are a very long time. I’m sorry that I didn’t met your expectations but I’m working hard to write the free tutorials faster. Please reconsider that I have to take care that all to code of the tutorials works correctly. I’m going to release part 3 next friday but I’m afraid it will disappoint you again by covering the user authentication/management of a real estate listing service first. I decided to cover these functions first because it doesn’t make any sense to go deeper into advanced search without the ability to submit properties. And property submission needs user registration functions. Kind regards, Lars

  4. Cornelius says

    I have been trying to follow you. I am getting a undefined method `search’ when performing searches and it points to the searches controller. I think I am getting the syntax wrong.

    Here’s the searches controller that has it wrong:

    def index
    @users = User.search(params[:search])
    end

    I have mines setup to use the User model, as I want it to pull information from User (will do other models later). Not sure if I have to add code to User model to get the controller to work. Just haven’t been able to follow what you have done well.

    • says

      Do you have thinking-sphinx added to your gemfile? And is ts running with rake ts:start? Has it been bundled? If you answered yes to all three then you should be able to run search on any models within your app.

      Getting results is another issue all together if you haven’t defined your indices as Lars pointed out.

      Hope this helps.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>