We’ve spent the last few weeks working on a new Rails app for a film and photography site and, as you might expect from my recent posts, we’re using Paperclip to handle file uploads.

Without any more effort than a script/install of Paperclip we get thumbnailing of images and PDFs, but what about thumbnailing the video files that will be uploaded to the site? Enter FFmpeg.

Installing FFmpeg

In case you’re not familiar with FFmpeg, it is an extremely powerful, open source recorder, converter and streamer of audio and video. Using it just to thumbnail videos is an almost criminal underuse of its capabilities.

And installation is a rare treat too: it’s easier on Windows than on Unix! Although there are no official binaries for Windows (and compiling your own would be painful) there are, thankfully, a number of pre-compiled options available. I decided to use this one, which includes everything needed to make the thumbnails.

1. Download and extract the latest build (I extracted the files to C:\Program Files\ffmpeg)
2. Add the folder to your PATH environment variable (this is optional, but if you don’t you’ll need to use the Paperclip :command_path option)

If you’re on Unix then you’re on your own! Although this page should help you get started…

The processor

Here’s the code:

module Paperclip
  class VideoThumbnail < Processor

    attr_accessor :time_offset, :geometry, :whiny

    def initialize(file, options = {}, attachment = nil)
      super
      @time_offset = options[:time_offset] || '-4'
      unless options[:geometry].nil? || (@geometry = Geometry.parse(options[:geometry])).nil?
        @geometry.width = (@geometry.width / 2.0).floor * 2.0
        @geometry.height = (@geometry.height / 2.0).floor * 2.0
        @geometry.modifier = ''
      end
      @whiny = options[:whiny].nil? ? true : options[:whiny]
      @basename = File.basename(file.path, File.extname(file.path))
    end

    def make
      dst = Tempfile.new([ @basename, 'jpg' ].compact.join("."))
      dst.binmode

      cmd = %Q[-itsoffset #{time_offset} -i "#{File.expand_path(file.path)}" -y -vcodec mjpeg -vframes 1 -an -f rawvideo ]
      cmd << "-s #{geometry.to_s} " unless geometry.nil?
      cmd << %Q["#{File.expand_path(dst.path)}"]

      begin
        success = Paperclip.run('ffmpeg', cmd)
      rescue PaperclipCommandLineError
        raise PaperclipError, "There was an error processing the thumbnail for #{@basename}" if whiny
      end
      dst
    end
  end
end

In order for Paperclip to automatically load the processor it should be saved in the RAILS_ROOT/lib/paperclip_processors directory.

A Paperclip processor must conform to the following rules:

• it must be defined within the Paperclip module as a subclass of Processor
• its constructor (if overriding the one provided by the base class) must accept the file to be processed, a hash of style options and the attachment instance that the file is for
• it must define a make instance method that returns a File or Tempfile containing the processed data

The VideoThumbnail processor accepts three options:

:time_offset

The number of seconds into the video to capture as a thumbnail (this should be a negative number and corresponds to the itsoffset option for FFmpeg).

:geometry

Accepts a wxh geometry string, ideally both the width and height should be even numbers however if they aren’t the processor will adjust them automatically.

:whiny

Determines whether or not thumbnailing errors are to be reported.

The make instance method of the thumbnailer actually does the work. The code itself is very straightforward:

• it creates a temporary file to store the thumbnail
• builds the options string to pass to FFmpeg
• calls FFmpeg, handling any errors
• it returns the temporary file (a JPEG image)

An example FFmpeg options string used by the thumbnailer looks like this:

-itsoffset -4 -i "C:/Temp/0001.avi" -y -vcodec mjpeg -vframes 1 -an -f rawvideo -s 320x240 "C:/Temp/0001.0001.jpg"

-itsoffset

The time offset (in seconds) to take the thumbnail.

-i

Specifies the input file (passed to the processor by Paperclip).

-y

Overwrites the output file if it exists.

-vcodec mjpeg

Specifies the output codec to use (in this case the mjpeg codec).

-vframes 1

The number of frames to output (we only want one frame as the output is a static image).

-an

Disables audio output.

-f rawvideo

Forces the output format to raw video (no encoding is necessary).

-s 320x240

Specifies the size of the thumbnail.

Patching Paperclip

Paperclip provides the :processors option that, unsurprisingly, allows you to specify the processors to run on a file. Today I submitted a patch that allows this option to accept a Proc that has now been applied to the plugin. By accepting a Proc it makes it possible to specify different processors depending on the type of file uploaded.

If you’re running on an older version of Paperclip and cannot upgrade to the latest version, you can monkey-patch it yourself:

module Paperclip
  class Attachment
    private
      def solidify_style_definitions
        @styles.each do |name, args|
          @styles[name][:geometry] = @styles[name][:geometry].call(instance) if @styles[name][:geometry].respond_to?(:call)
          @styles[name][:processors] = @styles[name][:processors].call(instance) if @styles[name][:processors].respond_to?(:call)
        end
      end
  end
end

In the model I’ve defined a video? method that returns true if the uploaded file is a video and then I use this is in the :processors option like this:

:processors => lambda { |a| a.video? ? [ :video_thumbnail ] : [ :thumbnail ] }

Handling file extensions

The final piece of the puzzle is telling Paperclip to use the correct file extension when downloading the thumbnails. Using the :extension interpolation provided by Paperclip won’t work because it uses the file extension of the originally uploaded file (if we upload a ‘.avi’ file then Paperclip would try to load thumbnails with a ‘.avi’ extension too).

My solution was to define a custom interpolation called :content_type_extension in an initializer like this:

Paperclip::Attachment.interpolations[:content_type_extension] = proc do |attachment, style_name|
  case
    when ((style = attachment.styles[style_name]) && !style[:format].blank?) then style[:format]
    when attachment.instance.video? && style_name.to_s == 'transcoded' then 'flv'
    when attachment.instance.video? && style_name.to_s != 'original' then 'jpg'
  else
    File.extname(attachment.original_filename).gsub(/^\.+/, "")
  end
end

This essentially does the same thing as the standard :extension interpolation but if the current style is not ‘original’ and the file is a video then it returns a ‘.jpg’ extension.

Putting it all together

So with a processor defined, Paperclip accepting Procs for the :processors option and a custom interpolation to handle the file extension, this is what a model would look like that provides video thumbnailing:

class Attachment < ActiveRecord::Base

  has_attached_file :asset,
                    :styles => { :small    => '36x36#',
                                 :medium   => '72x72#',
                                 :large    => '115x115#' },
                    :url => '/:class/:id/:style.:content_type_extension',
                    :path => ':rails_root/assets/:id_partition/:style.:content_type_extension',
                    :processors => lambda { |a| a.video? ? [ :video_thumbnail ] : [ :thumbnail ] }

  def video?
    [ 'application/x-mp4',
      'video/mpeg',
      'video/quicktime',
      'video/x-la-asf',
      'video/x-ms-asf',
      'video/x-msvideo',
      'video/x-sgi-movie',
      'video/x-flv',
      'flv-application/octet-stream',
      'video/3gpp',
      'video/3gpp2',
      'video/3gpp-tt',
      'video/BMPEG',
      'video/BT656',
      'video/CelB',
      'video/DV',
      'video/H261',
      'video/H263',
      'video/H263-1998',
      'video/H263-2000',
      'video/H264',
      'video/JPEG',
      'video/MJ2',
      'video/MP1S',
      'video/MP2P',
      'video/MP2T',
      'video/mp4',
      'video/MP4V-ES',
      'video/MPV',
      'video/mpeg4',
      'video/mpeg4-generic',
      'video/nv',
      'video/parityfec',
      'video/pointer',
      'video/raw',
      'video/rtx' ].include?(asset.content_type)
  end

end

In this example the video? method contains the content types that are considered a video file. In your application you will likely want to put this in a configuration file of some sort.

So there you have it: video thumbnailing using FFmpeg and Paperclip. As part of the same project we’ve also managed to implement transcoding video uploads to FLV format using FFmpeg in another Paperclip processor… but that’s a story for another time!