h1. Using Gettext To Translate Your Rails Application
p{color: red}. The version of this document is optimized for Ruby-GetText 1.9.0 and Rails 1.2.
h2. Introduction
h3. Stand on the shoulders of Giants
"Gettext":http://www.gnu.org/software/gettext/ is a great tool for translating user interfaces of applications into different languages. It has been around for a long time and is very well established for this task. Gettext helps you to open up your application to a much wider user base than you would normaly reach in only one language. Since it is used in many open source projects it has a lot of useful tools you can use to your own advantage. It would be possible to "roll your own" solution, but this would consume a large amount of time. Time you would lose for the development of your app. And this is not something you would want, right?
So in this tutorial I am going to show you how to effectively use Gettext for all your translation needs in your Ruby on Rails application.
h3. What can Gettext do for you?
Here is a quick explanation of Gettext for those of you who haven't worked with it, yet. Gettext is a set of tools that provide help when translating strings in your software. It gives you (straight from the "gettext manual":http://www.gnu.org/software/gettext/manual/html_node/gettext_2.html#SEC2):
* A set of conventions about how programs should be written to support message catalogs.
* A directory and file naming organization for the message catalogs themselves.
* A runtime library supporting the retrieval of translated messages.
* A few stand-alone programs to massage in various ways the sets of translatable strings, or already translated strings.
The only thing you have to do to your program sources is wrap all translatable strings with a method call: gettext(text) or shorter _(text). Example:
Without gettext:
notice = 'Thank you for buying our product.'
With gettext:
notice = _('Thank you for buying our product.')
If you have wrapped everything between the method call Gettext will provide you with the tools to extract these messages (called a harvester) from your source code and save the results to a portable file format. A runtime library will allow you to display a translated message whenever _() is called.
In essence this is what Gettext does. It can do a couple of more things (like update / merge message catalogs, handling plurals, ...) but essentially it tries to make it as easy and as unobtrusive as possible to translate the language strings in your source code and then get out of your way.
h2. Preperation is everything -- use UTF-8 everywhere
h3. Edit your files with UTF-8 only
The W3C has an awesome page with all you need to know about character sets (choosing, declaring, serving, editing, ...) and other important stuff concerning internationalization on their site called "W3C I18N Topic Index":http://www.w3.org/International/resource-index.html. There you will find a lot of good help and suggestions on everything you need to know about general internationalization topics. You really want to spend at least a couple of hours reading through the resources they offer or link to.
If you happen to use VIM like I do you can put these two lines in your .vimrc and it will from then on treat your files as UTF-8 and convert everything to UTF-8 whenever you save a file:
set encoding=utf8
set fileencoding=utf8
If your editor of choice doesn't provide support for UTF-8 it is probably time to get a new one ;)
h3. Feed your database with UTF-8
If I need a database I usually go for MySQL. At least most of the time. Since version 4.1.x MySQL has very good support for different character sets. Please read the extensive "MySQL Character Set Support":http://www.quepublishing.com/articles/printerfriendly.asp?p=328641&rl=1 article. It will give you a very good idea about the kind of support MySQL has to offer.
Here is a table definition I use in a current project:
CREATE TABLE `pages` (
`id` int(10) unsigned NOT NULL auto_increment,
`title` varchar(255) default NULL,
`keywords` varchar(255) default NULL,
`description` varchar(255) default NULL,
`block1` text,
`block2` text,
`block3` text,
`block4` text,
`block5` text,
`lang` varchar(10) NOT NULL default 'en_EN',
`category` varchar(255) default NULL,
`path` varchar(255) default '/',
`updated_at` datetime default NULL,
`created_at` datetime default NULL,
`published_at` datetime default NULL,
`layout` varchar(255) default 'main.rhtml',
`template` varchar(255) default NULL,
`access` tinyint(3) unsigned default '3',
`version` int(10) unsigned default '1',
`is_published` tinyint(3) unsigned default '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;
There is nothing really special about it except the DEFAULT CHARSET=utf8 in the last line. It will make MySQL use UTF-8 for every field that holds text like varchar and text.
Add the following line to your database.yml to connect with UTF-8.
encoding: UTF8
If you use another maybe more mature and feature rich database like PostgreSQL UTF-8 support should also be available. Just make sure you configure everything upfront, because it might save you time later on.
If you still think you are going to tack on support for UTF-8 later you might reconsider after reading David's and Jamis' experiences: "Forty-four grueling hours (or Welcome to 37s!) by David Heinemeier Hansson":http://www.loudthinking.com/arc/000415.html and "On the job by Jamis Buck":http://weblog.jamisbuck.org/2005/3/5/on-the-job.
If for any reason you need to use anything other than UTF-8 you might want to know that you can use the Iconv module to convert between different character sets. This module should be installed by default on most distributions. If it is not it is usually easy to get. Just be sure to get it if you need charcter set conversion. On Debian you might need a apt-get install libiconv-ruby to get started. Here is a little example stolen from "Pickaxe 2":http://pragmaticprogrammer.com/titles/ruby/ page 686:
Example: Convert olé from UTF-8 to ISO-8859-1
$ irb -riconv
>> Iconv.conv('ISO-8859-1', 'UTF-8', "ol\303\251")
=> "ol\351"
h3. Installing Ruby-GetText
After everything in your setup will accept UTF-8 we can go ahead and install Gettext support in Ruby. We will use the "Ruby-GetText-Package":http://gettext.rubyforge.org/ which is a Ruby implementation of the Gettext interface that also has some c-bindings for the Locale.
If you have rubygems, you can easily install Ruby-GetText:
$ gem install gettext
Select which gem to install for your platform (...)
1. gettext 1.9.0 (ruby)
2. gettext 1.9.0 (mswin32)
...
>
Be sure to choose option 2 if you want Ruby-GetText to work on native Windows (with "One-Click Ruby Installer":http://rubyforge.org/projects/rubyinstaller/) as you are most likely not able to compile otherwise. For all other platforms use 1.
Alternative: If you need the source you can get it from the author's site: "Ruby-GetText-Package":http://gettext.rubyforge.org/. Besides "installation instructions":http://ponx.s5.xrea.com/hiki/ruby-gettext.html#Install you will find a small "HOWTO":http://ponx.s5.xrea.com/hiki/ruby-gettext-howto.html, "API Reference":http://ponx.s5.xrea.com/hiki/ruby-gettext-api.html and documentation on "how to use provided tools":http://ponx.s5.xrea.com/hiki/ruby-gettext-tools.html.
Check the sample Rails application that is delivered with Ruby-GetText. If you installed via RubyGems it is most likely here: /usr/lib/ruby/gems/1.8/gems/gettext-1.9.0/samples/rails/, but it could be somewhere else depending on where your Ruby is installed. Later I will call that path $RUBYGETTEXT_RAILS_SAMPLE. So keep this in mind when you read.
h3. Create the "po" directory structure
Create a po dir right in your $RAILS_ROOT. In it you will create a subdir for every language/dialect combination (actually the second part of this abbreviation stands for "geographic region", but I will just call it dialect throughout this document). If you don't know the right code for your language you can seek help at the "Language section of the W3C I18N Topic Index":http://www.w3.org/International/resource-index.html#lang.
Ultimately your directory structure is going to look like this:
simplepages@colinux: /home/simplepages/rails/po:
$ d -T
/home/simplepages/rails/po/:
|-myapp.pot
|-de_DE/:
| `-myapp.po
|-en_GB/:
| `-myapp.po
|-en_US/:
| `-myapp.po
|-fr_FR/:
| `-myapp.po
|-fr_CH/:
| `-myapp.po
`-ja/:
`-myapp.po
The myapp.pot file is the original file created by the rgettext script introduced later. The po files will contain the translation messages that your application is going to use depending on the language that is requested.
h3. Convert pofiles to mofiles
After creating pofiles, you need to convert them to mofiles.
If you haven't yet, please read the the "GNU Gettext manual with the explanation of what mo and po stands for":http://www.gnu.org/software/gettext/manual/html_node/gettext_5.html#SEC5.
Add the code below to Rakefile:
simplepages@colinux: /home/simplepages/rails/Rakefile
require 'gettext/utils'
desc "Create mo-files for L10n"
task :makemo do
GetText.create_mofiles(true, "po", "locale")
end
Then,
$ rake makemo
It will create locale directories and subdirectroies such as:
simplepages@colinux: /home/simplepages/rails/locale:
$ d -T
/home/simplepages/rails/locale/:
|-de_DE/:
| `-LC_MESSAGES/:
| |-myapp.mo
|-en_GB/:
| `-LC_MESSAGES/:
| |-myapp.mo
|-en_US/:
| `-LC_MESSAGES/:
| |-myapp.mo
|-fr_FR/:
| `-LC_MESSAGES/:
| |-myapp.mo
|-fr_CH/:
| `-LC_MESSAGES/:
| |-myapp.mo
`-ja/:
`-LC_MESSAGES/:
|-myapp.mo
These files are used by Ruby-GetText. You don't need to touch these files.
h2. Tools of trade
h3. Gettext
You will need to install Gettext. On Debian I would do apt-get install gettext to do that. It contains a couple of tools that will be handy later on, most importantly msgmerge which can merge different po files so updating your message files will be a snap and msginitwhich can set default values to the header of pofile in your locale.
h3. Ruby-GetText and rgettext
After you have installed Ruby-GetText and tools you should able to call rgettext on the command line. rgettext is a replacement for xgettext which comes with the main Gettext application. Beyond xgettext, rgettext supports not only ruby scripts(.rb) but also ERB files (.rhtml), ActiveRecord(.rb) directly.
h4. ActiveRecord support
rgettext extracts all the table names and field names within subclasses of ActiveRecord::Base.
Notice: You need to run your database server and configure the config/database.xml correctly before executing rgettext.
h3. poEdit
"poEdit":http://www.poedit.net is a great tool to manage and edit your translations. It gives you a nice graphical frontend to translate all your messages. It is available for many different platforms. A nice side effect about using an easy to use GUI tool is that you can tell non-programmers to install it, open the po file and just start translating. They might have nothing to do with the coding in your project but will be able to help you translate your software. So if your grandma can translate English to Chinese she can maybe help you with your software project :)
h2. Collecting messages
Add the code below to Rakefile:
simplepages@colinux: /home/simplepages/rails/Rakefile
desc "Update pot/po files to match new version."
task :updatepo do
MY_APP_TEXT_DOMAIN = "myapp"
MY_APP_VERSION = "myapp 1.1.0"
GetText.update_pofiles(YOUR_APP_TEXT_DOMAIN,
Dir.glob("{app,lib}/**/*.{rb,rhtml}"),
MY_APP_VERSION)
end
Running this task will either create or update your po/myapp.pot and po/*/myapp.po files in the relevant directories. It will go through all the important directories of your rails app and harvest all the Gettext strings in files ending in *.rb, *.rhtml.
$ rake updatepo
h3. Translating and compiling with and without poEdit
After you have successfully harvested your files you should have a myapp.po file in every locale dir. Now you need to translate them. Since the myapp.po files are mere text files you could just use your favourite text editors to translate them. Given that your text editor can edit in UTF-8 mode and you know the escaping rules of Gettext this is all you actually need. Open the file, translate the text and save it. After you have saved the file compile it. Gettext doesn't work with the text files (myapp.po) directly. It wants a compiled version of it (myapp.mo). Use the rake makemo command to compile:
simplepages@colinux: /home/simplepages/rails/po/de_DE:
$ ls
myapp.po
simplepages@colinux: /home/simplepages/rails:
$ rake makemo
simplepages@colinux: /home/simplepages/rails/locale/de_DE/LC_MESSAGES:
$ ls
myapp.mo
However, it is way more comfortable to use an application like poEdit for this. With poEdit you can also easily open up the myapp.po file. It will give you a nice side by side view your original strings and the translated version, telling what is already translated and what is not. You click on a message and start translating it in a special field. Hit the save button and poEdit will automatically compile the myapp.mo file for you (check the preferences if it doesn't do it by default). That's it. With a compiled myapp.mo you can start to teach your rails app how to translate your user interface.
See "Documents for Translators":http://ponx.s5.xrea.com/hiki/ruby-gettext-translate.html for more details to translate the po-file.
When creating a po file it is useful to include header information at the top of the po file. This adds useful information such as character set, last translator etc. If you are not using poEdit add something similar to the top of the po file.
msgid ""
msgstr ""
"POT-Creation-Date: 2007-02-19 17:15-0000\n"
"PO-Revision-Date: 2003-04-03 10:49--500\n"
"Last-Translator: Alastair Brunton \n"
"Language-Team: fr_FR \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
h2. Implementing Ruby-GetText into your rails app
By now you should have a translated and compiled myapp.mo in your locale dir. For example my German translation of SimplePages is at $RAILS_ROOT/locale/de_DE/LC_MESSAGES/myapp.mo.
h3. Including Ruby-GetText
Edit application.rb to bind textdomain to your application.
simplepages@colinux: /home/simplepages/rails/app/controllers/application.rb:
require 'gettext/rails'
class ApplicationController < ActionController::Base
init_gettext "myapp"
#init_gettext "myapp", "UTF-8", "text/html" # <= Also you can set charset and content_type.
end
In this sample, the textdomain name is "myapp". Replace it as you like to fit your application. Maybe you want to have different textdomains for your site and the admin section.
h3. Selecting the scope of your textdomain
# If you call bindtextdomain in ApplicationControler, the textdomain applies to the entire application.
# If you call bindtextdomain in any other controller with a different textdomain, this textdomain only applies to this specific controller. For example if you call a different textdomain in myapp_controller.rb it will only be used in myapp_controller.rb.
The textdomains are applied to each controller/view/model.
h3. Choosing the right language on every request
Since we are developing a web application we want to be able to choose the current language by request. Additionally we might want to offer the user the possibilty to choose the language from a menu.
Ruby-GetText chooses the current language by following these rules in the given order:
# the first value passed to the 'locale' parameter of GetText.bindtextdomain method call
# 'lang' value of QUERY_STRING
# 'lang' value of the Cookie
# the value of HTTP_ACCEPT_LANGUAGE
# or default 'en' (English).
The script $RUBYGETTEXT_RAILS_SAMPLE/vendor/plugins/lang_helper.rb is a sample that selects locale using the cookie value of the user.
It may be useful to implement a simple controller action to change between languages eg.
class CookieController < ApplicationController
def set_cookie
code = params[:id]
cookies[:lang] =
{
:value => code,
:expires => Time.now + 1.year,
:path => '/'
}
redirect_to home_url
end
end
So /cookie/set_cookie/fr_FR would change the language to French.
h3. Using the Locale in your templates
Depending on the selected locale you will want to customize the language and character set in your templates.
File: $RAILS_ROOT/app/view/layouts/main.rhtml
<%= @page["title"] %>
...
h3. Have your own textdomain for plugin applications
If you are a plugin developer and want to have your own textdomain, you need to separate the Class/Module from ActionView::Base/ApplicationController.
simplepages@colinux: /home/simplepages/rails/vendor/plugins/gettext/lib/gettext_plugin.rb:
require 'gettext/rails'
module LangHelper
# If you need to bind yet another textdomain to your plugin.
# Separate the name space from ActionView::Base/ApplicationController.
class YetanotherTextDomain
include GetText::Rails
def initialize
# You need to call bindtextdomain in an instance of ActionView::Base.
# The locale is used a same values which define ApplicationController#init_gettext instead of
# the textdomain.
bindtextdomain("gettext_plugin")
end
def show_language(actionview)
langs = ["en"] + Dir.glob(File.join(RAILS_ROOT,"locale/*")).collect{|item| File.basename(item)}
langs.delete("CVS")
langs.uniq!
ret = "" + _("Select locale") + "
"
langs.sort.each do |lang|
ret << actionview.link_to("[#{lang}]", :action => "cookie_locale", :lang => lang)
end
ret
end
def cookie_locale(cookies, flash, params)
cookies["lang"] = params["lang"]
flash[:notice] = _('Cookie "lang" is set: %s') % params["lang"]
end
end
# This function shows supported languages with link to set cookie
# action (cookie_locale).
def show_language
YetanotherTextDomain.new.show_language(self)
end
# This function is called when the language link is set.
def cookie_locale
YetanotherTextDomain.new.cookie_locale(cookies, flash, params)
redirect_to :action => 'list'
end
end
Simply put gettext_plugin.po into the po directory.
h2. Conclusion
That's it. If you have your translated message catalogs (myapp.mo) in all the right places your application should show your message strings in your favourite language.
You can now easily start to translate your application into all the different languages you want. I hope this guide helps you to get started. There are certainly many more aspects of internationalization that you will have to learn and apply. Remember that this is only one of many possible ways to do it. If you find any mistakes, shortcomings or have good suggestions on how to improve this guide I would be more than happy to hear from you.
If you want you can download the "original Textile document":http://www.digitale-wertschoepfung.de/artikel/gettext/gettext.txt, make modifications and send them back to me. I will be sure to include you in the credits section.
Sascha Ebach
"se at digitale-wertschoepfung dot de":mailto:se at digitale-wertschoepfung dot de
h3. Credits
* "David Heinemeier Hansson":http://www.loudthinking.com for creating the best web application framework that has ever existed P-E-R-I-O-D
* Masao Mutoh - The Author of Ruby-GetText: "mutoh at highway dot ne dot jp":mailto:mutoh at highway dot ne dot jp. Thanks to Masao for extending the old version and making it more flexible.
* "Alastair Brunton":http://www.simplyexcited.co.uk for providing some sorely needed modifications to this article.
h3. Author
Sascha Ebach is the owner and lead developer of a small "web design and development shop":http://www.digitale-wertschoepfung.de "Digitale Wertschöpfung":http://www.digitale-wertschoepfung.de in Cologne, Germany. Together with his two partners he develops and designs complete online solutions for small to medium sized businesses. Up until the surfacing of Rails he used to develop everything in PHP although he has already fallen deeply in love with Ruby since version 1.6.2 came out. For him it is very clear that Rails and Ruby will be *The Future Way* of developing web applications and he already looks forward to the day when he has ported his last line of PHP to Ruby.
h2. Appendix A: Downloads
h3. Used files
Download the archive with files and scripts I use and talk about in this guide. The file includes the complete skeleton of files that you need to get started.
/home/simplepages/using-gettext-with-rails/:
|-app/:
| `-controllers/:
| `-application.rb (the ApplicationController with the .init_gettext method)
`-po/: (sample directory structure)
|-de_DE/:
|-en_GB/:
`-en_US/:
"Download using-gettext-with-rails.tgz":http://www.digitale-wertschoepfung.de/artikel/gettext/using-gettext-with-rails.tgz