memcached is a popular caching solution in Rails community. cache_fu is an excellent plugin that makes it a breeze to deal with memcached. However going through the README of this plugin does not tell you much if you are new to this plugin.

Basics of memcached

memcached is a big topic. Here is an excellent resource to get started. Here I am going to discuss some basic things about memcached.

memcached is extremely popular and stable. Sites like facebook and flick use memcached.

memcached is like a big hash table. What you store is name value pair. The key must of 250 characters or less. The data could be text or binary. The max size of the data is 1MB . While storing the value you can mention when the data should be expired. The maximum duration for which data can be kept is 30 days .

memcached does not support namespacing natively. If you want namespacing then add namespace to your key.

When you start memcached then the default port it uses is 11211. The default memory is 64MB. You can find out the memcached version and more options by running

memcaced -h

How does expiration work in memcached

memcached uses a lazy expiration, which means it uses no extra cpu expiring items. When an item is requested (a get request) it checks the expiration time to see if the item is still valid before returning it to the client.

Similarly when adding a new item to the cache, if the cache is full, it will look at for expired items to replace before replacing the least used items in the cache.

cache_fu

cache_fu is an excellent plugin to manage memcached. To explore cache_fu I will create a simple application.

I am using ruby 1.8.6 and Rails 2.3 .

rails cache_fu_lab
cd cache_fu_lab
ruby script/generate scaffold User name:string email:string
rake db:create
rake db:migrate

In development environment caching is turned off by default. In order to turn caching ON open ‘config/environements/development.rb’ and change the value from ‘false’ to ‘true’ for ‘config.action_controller.perform_caching’

config.action_controller.perform_caching             = true #false

Install memcache-client. Please note that ActiveSupport ships with memcache-client too. If you look under vendor directory you will find memcache-client-1.6.5 .

> sudo gem install memcache-client

Install plugin

ruby script/plugin install git://github.com/defunkt/cache_fu.git 

Now you can manage memcached by following rake tasks

rake memcached:start
rake memcached:stop
rake memcached:restart

Let’s create a user

>> User.create(:email => 'john@example.com', :name => 'John')
  User Create (0.8ms)   INSERT INTO "users" ("name", "updated_at", "email", "created_at") VALUES('John', '2009-08-01 17:27:49', 'john@example.com', '2009-08-01 17:27:49')
=> #<User id: 1, name: "John", email: "john@example.com", created_at: "2009-08-01 17:27:49", updated_at: "2009-08-01 17:27:49">

Add caching capability to the model.

#user.rb
class User < ActiveRecord::Base
  acts_as_cached
end

In users_controller.rb file replace this line

@user = User.find(params[:id])

with this line

@user = User.get_cache(params[:id])

Now visit http://localhost:3000/users/1

You will see something like this in the log

> Got User:1 from cache. (0.00073)
  User Load (0.2ms)   SELECT * FROM "users" WHERE ("users"."id" = 1) 
> Set User:1 to cache. (0.00107)

Although the first line says “Got”. That was actually a miss. That is why sql statement was executed and the result of the sql was stored in memcached. If you refresh the page then you will see something like this in the log.

==> Got User:1 from cache. (0.00041)

This time you won’t see any sql statements. That is because this data was retrieved from cache.

memcached server is running and it’s doing its job. It would be nice if we have a client using which we could get and set data on the server. This can be done like this.

>ruby script/console
> User.find(1) # to check that we have a record with id 1
> CACHE = MemCache.new 'localhost:11211'
=> MemCache: 1 servers, ns: nil, ro: false
>> CACHE.active?
=> true
>> CACHE.get('app-development:User:1')
=> #<User id: 1, name: "John", email: "john@example.com", created_at: "2009-08-01 17:27:49", updated_at: "2009-08-01 17:27:49">
>> CACHE.get('app-development:User:100')
=> nil

As you can see above we created an instance of MemCache with localhost:11211. active? method is called to see if this server is active or not. And then we passed a key to server to see if there is a record for app-development:User:1. And we got our record.

You might be wondering what’s up with the key ‘app-development:User:1’. Well ‘app’ is the namespace defined in config/memcached.yml. cache_fu automatically adds Rails.env value to the namespace. In this way we got ‘app-development’. Since we stored a user instance in the cache which had the id of ‘1’, the full key formed by cache_fu was ‘app-development:User:1’

method caches

I added a method called info on user model.

def self.info(id)
   @user = self.find(id)
   "name: #{@user.name} email:#{@user.email}" 
end

If you want to cache the output of the method then you can invoke it like this

 <%=h User.caches(:info,:with => params[:id]) %>

If the params[:id] is ‘1’ then cache_fu will save the result with key ‘User:info:1’

fragment caching

If want to use memcached for fragment caching then you need to open up config/memcached.yml and change

fragments: false

to

fragments: true

Example of fragment caching

<% cache('user_info') do %>
<p>
  <b>Name:</b>
  <%=h User.find(params[:id]).name %>
</p>
<% end %>

First time when you visit the page you will get this message in the log

Cached fragment hit: views/user_info (2.4ms)
  User Load (0.2ms)   SELECT * FROM "users" WHERE ("users"."id" = 1) 
Cached fragment miss: views/user_info (0.5ms)

It was a miss. And that is why you see sql statement. Subsequent refreshing of the same page will have following message in the log

Cached fragment hit: views/user_info (0.7ms)

If you want to view the result using memcached-client then you have to follow the same procedure.

>> CACHE = MemCache.new 'localhost:11211'
=> MemCache: 1 servers, ns: nil, ro: false
>> CACHE.get('app-development:views/user_info')
=> "\n<p>\n  <b>Name:</b>\n  John\n</p>\n" 

action caching

As of this writing action caching in cache_fu is broken . Hopefully it will be fixed soon. Until the follow the instructions mentioned at the end of the blog to make it work.

action caching internally uses fragment caching with an around-filter. In the last section when we change the settings to use fragment caching we also automatically turned on memcaching for action caching. Let’s see it in action

class UsersController < ApplicationController
  caches_action :index
  def index
    @users = User.all
    respond_to do |format|
      format.html
    end
  end
end

In the log first time we will see the sql statement. Subsequently we will not see any sql statements. Instead we’ll get a message like this

Cached fragment hit: views/localhost:3000/users (0.1ms)

If you want to verify this cache using the client then you can do like this.

> ruby script/console
>> CACHE = MemCache.new 'localhost:11211'
=> MemCache: 1 servers, ns: nil, ro: false
>> CACHE.active?
=> true
>> CACHE.get('app-development:views/localhost:3000/users')
=> "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"\n       "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n\n<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">\n<head>\n  <meta http-equiv="content-type" content="text/html;charset=UTF-8" />\n  <title>Users: index</title>\n  <link href="/stylesheets/scaffold.css?1249146947" media="screen" rel="stylesheet" type="text/css" />\n</head>\n<body>\n\n<p style="color: green"></p>\n\n<h1>Listing users</h1>\n\n<table>\n  <tr>\n    <th>Name</th>\n    <th>Email</th>\n  </tr>\n\n\n  <tr>\n    <td>John</td>\n    <td>john@example.com</td>\n    <td><a href="/users/1">Show</a></td>\n    <td><a href="/users/1/edit">Edit</a></td>\n    <td><a href="/users/1" onclick="if (confirm('Are you sure?')) { var f = document.createElement('form'); f.style.display = 'none'; this.parentNode.appendChild(f); f.method = 'POST'; f.action = this.href;var m = document.createElement('input'); m.setAttribute('type', 'hidden'); m.setAttribute('name', '_method'); m.setAttribute('value', 'delete'); f.appendChild(m);var s = document.createElement('input'); s.setAttribute('type', 'hidden'); s.setAttribute('name', 'authenticity_token'); s.setAttribute('value', 'mqPwXqFLN0pIM1d5m3QfaeWBRqEb/36Dkh4uErIfyG4='); f.appendChild(s);f.submit(); };return false;">Destroy</a></td>\n  </tr>\n\n</table>\n\n<br />\n\n<a href="/users/new">New user</a>\n\n</body>\n</html>\n" 

Storing session information on memcached

By default Rails now stores session data as a cookie on client browser. However, if you want memcached to store session data then open config/memcache.yml and change the value for sessions from ‘false’ to ‘true’. That’s it. Now all session data will be stored on memcached.

Expiring cached items

I have action caching for ‘index’ method which looks like this.

class UsersController < ApplicationController  
  caches_action :index
  def index
    @users = User.all
    respond_to do |format|
      format.html
    end
  end
end

In order to expire the old pages we need to have sweepeers.

mkdir app/sweepers

Create a file called user_sweeper.rb with following content.

class UserSweeper < ActionController::Caching::Sweeper

  observe User

  def after_create(user)
    expire_cache_for(user)
  end

  def after_update(user)
    expire_cache_for(user)
  end

  def after_destroy(user)
    expire_cache_for(user)
  end

  private

  def expire_cache_for(user)
    #expire the index page that is action cached
    expire_fragment(:controller => 'users', :action => 'index')
  end
end

Add app/sweepers to the load path.

# environment.rb
Rails::Initializer.run do |config|
  config.load_paths += %W( #{RAILS_ROOT}/app/sweepers )
  ...
end

Add cache_sweeper in the controller.

class UsersController < ApplicationController  
  caches_action :index
  cache_sweeper :user_sweeper, :only => [:create,:update,:destroy]
  def index
    @users = User.all
    respond_to do |format|
      format.html
    end
  end
end

Now if you create a new user and then visit the page http://localhost:3000/users you will see the updated list.

Now let’s look at how to expire_cache for data cached for model. Caching is done like this.

def show
  @user = User.get_cache(params[:id])
  respond_to do |format|
    format.html
  end
end

In the sweeper, following line should be added for the method def after_update

User.expire_cache(user.id)

Now let’s look at how to expire fragment cache. Code is

<% cache('user_info') do %>
<p>
  <b>Name:</b>
  <%=h User.find(params[:id]).name %>
</p>
<% end %>

Add following line to method after_update in sweeper to expire the fragment cache whenever data is updated

expire_fragment('user_info')

Conclusion

cache_fu is an excellent plugin which takes the pain away in dealing with memcached. Hopefully this article will help people get started with cache_fu without much pain.