Unit Testing with Tire and Elastic Search

I’ve been working a lot recently with ElasticSearch and the fantastic Tire gem.  Trying to stay true to my test first philosophy, I spent some time trying to figure out how to get Tire to work just right with RSpec, but encountered a few issues along the way (which were thankfully all solvable).  

The first issue I encountered with using Tire in RSpec was the need to clean out the elasticsearch index between every test.  This was easily fixed by deleting the index before each run in the spec_helper.rb as below:

config.before :each do

  User.index.delete

end

This solved the problem of clearing out the data between tests, but I quickly ran into another issue.  ElasticSearch out of the box will index new documents even without any mapping being specified. However, it will assume everything is a string and won’t apply any custom analyzers.  In my particular case I use geo_point fields to do location based search (I’ll do another article at some point in the future to talk about that).  Since I went ahead and deleted my User index after each run, any run except for the first would not have the latitude and longitude set as geo_point fields and all my searches against Users in my tests would fail with the following error: Tire::Search::SearchRequestFailed.  This was easily remedied by changing my spec_helper once again to:

config.before :each do

  User.index.delete

  User.create_elasticsearch_index

end

Now between runs the index is deleted and then the mapping from tire is passed again to elasticsearch, meaning all my field types are correct and any analyzers I intend to use are set correctly.  Yay!

Now that I got that working, I noticed that none of my documents were showing up in the index after calling save.  This had to do with the fact that ElasticSearch only refreshes its index once every second and our tests run a little quicker than that.  Short answer is to force a refresh of the object you just saved to your database, so it gets pushed out to elasticsearch.  Like this:

user.index.refresh

Now my setup was working great until I realized that now I was deleting any data from my development search index whenever my tests ran (oops) since they both run again my local machine.  This was a relatively easy fix and just involved setting the index name to something other than users when running tests.  I did this by setting the index name again in spec_helper, but in the before :all block.

config.before :all do

  User.index_name(‘test_’ + User.model_name.plural)

end

This just prefixes the index name with “test_” for any User index created by RSpec.

Finally, I wanted to try and speed things up a bit.  My 81 tests were taking about 16 seconds to run now on my late 2010 MacBook Air.  A quick thought I had was to index everything to memory instead of the disk while in the test environment.  I did it like this:

settings :index => {

    :store => {

      :type => Rails.env.test? ? :memory : :niofs

    }

  } do

  mapping do

    #mapping here

  end

end

Unfortunately this only managed to shave a second off the total time to run, but it’s a start.  I would like to make it faster, but this will have to do for now.  Anyone else have any good optimization tricks?  Share in the comments!

9 notes

A week ago I took a trip out to SF to attend a conference at the Googleplex.  The company I work for, Signpost, is a Google Ventures company so they had invited us out there to see some of the the things the crazy minds there were working on.  All in all it was a great conference, but the real highlight of the trip for me was San Francisco and its amazing food.  One particularly good place we stopped for brunch was Mission Beach Cafe.

I ordered the dungeness crab and shrimp eggs benedict and a side of candied bacon (note: I think their menu changes).  The eggs benedict was simply amazing and cooked to perfection.  The egg was perfectly runny and the crab was absolutely delicious.  The crunchiness of the crab and shrimp combined with the oozy goodness of the egg went perfectly together.  I savored every bite.  

The bacon came out a little later in the meal due to some ordering snafu, but I was fine with that cause this bacon was more like dessert.  The candied glaze on top of the not too crunchy, not too soft bacon was really nice and sweet.  It was bacon unlike any I’ve ever tasted and it was probably the best.  I had split the bacon with one of my co-workers and had to hold myself back from eating his share.  If I had to describe the taste, it was something similar to bacon glazed with maple syrup, but that doesn’t remotely do it justice.  It was incredibly delicious and all bacon will now pale in comparison.

I hope you all enjoyed my first food post (I honestly wish I knew I was gonna write this blog before I had the meal, I would have taken notes).  Anyways off to code some things before bed…

0 notes

Hello World!

So being a half programming blog (the other half being food) I felt obliged to keep with tradition and entitle my first post “Hello World!”.  I’m gonna use this time to tell ya’ll a little bit about myself and the kinds of things you’ll see here.

My name is Jason Falk and I currently live in NYC (up until recently, I was one of those people that declared NYC the best city without even really experiencing many others).  I’m a Software Engineer at Signpost and a self described foodie.  On the programming end I’m traditionally a Java Developer, but recently made the jump to Ruby on Rails (and not looking back).  On top of your average web development I work mostly in the area of search, using both SOLR and more recently ElasticSearch to build all sorts of things.  

As for food.  Well, I’ll eat almost anything and it was food that actually gave me the impetus to start this blog.  While on a recent trip to SF (my new favorite city) for a conference at the Googleplex (wow what a cool place) i brought along my DSLR (Canon T2i) and snapped some pics of all the tasty things I ate along the way and decided I would like to share them (the pics, not the food).  

Anyways.  Looking forward to sharing these things with all of you and hope to learn stuff along the way as well.

0 notes