Today I had to create an XML with ~ 7000 records.

The first role looked like this:


Controller
def all_articles
@articles = Article.find_all
render :layout => false
end

View - articles.rxml
xml.instruct! :xml, :version => '1.0'
xml.root do
xml.articles do
@articles.each do |article|
xml.article do
xml.id article.id
xml.title article.title
end
end
end
end

The size of the XML was ~ 800kb and it took 7 seconds to render.
Horror, ah?

The truth is, after thinking, I don't really need an XML in this specific case. All the data is going to be read by a JavaScript that I can modify to get any format I like.

The solution:


Controller
def all_articles
ActiveRecord::Base.connection.execute("SET SESSION group_concat_max_len = 999999999;")
@articles_big_csv = ActiveRecord::Base.connection.execute("select group_concat(articles.id,'>',articles.name SEPARATOR '<') from articles").fetch_row.first render :layout => false
end

View - aricles.rxml
xml.instruct! :xml, :version => '1.0'
xml.root do
xml.articles @articles_big_csv
end

The output of "select group_concat(articles.id,'>',articles.name SEPARATOR '<') from articles" is in the format of "ARTICLE ID>ARTICLE TITLENEXT ARTICLE TITLE..."
(I can use the "<>" as separators since I dont allow them in an article name; else I could use any sepeartors I like)

Later, on the parsing JavaScript side I will split the string by '<' to get the rows and each row I will split by '>' to get it's columns.

The size of this output is now 200kb and rails can render it 11(req/s)

(...if you are lucky and your server gzips everything it goes down to 80kb)

What did we do?
1. Smallest size: we didn't use XML standards but a simple primitive string with separators.
2. Fastest tool: we did all the work in MySql which is much faster than Ruby.

Kudos to Roman Krom for the 'let sql do the work for you' philosophy
Kudos to Boris Peterbard for teaching me the group_concat magic