Localization
Chapter 7. Localization
- Table of Contents
- 7.1. Locale Support
- 7.2. Multibyte Support
- 7.2.1. Supported character set encodings
- 7.2.2. Setting the Encoding
- 7.2.3. Automatic encoding conversion between server and client
- 7.2.4. What happens if the translation is not possible?
- 7.2.5. References
- 7.2.6. History
- 7.2.7. WIN1250 on Windows/ODBC
- 7.3. Single-byte character set recoding
PostgreSQL supports localization with three approaches:
Using the locale features of the operating system to provide locale-specific collation order, number formatting, translated messages, and other aspects.
Using explicit multiple-byte character sets defined in the PostgreSQL server to support languages that require more characters than will fit into a single byte, and to provide character set recoding between client and server. The number of supported character sets is fixed at the time the server is compiled, and internal operations such as string comparisons require expansion of each character into a 32-bit word.
Single byte character recoding provides a more light-weight solution for users of multiple, yet single-byte character sets.
7.1. Locale Support
Locale support refers to an application respecting cultural preferences regarding alphabets, sorting, number formatting, etc. PostgreSQL uses the standard ISO C and POSIX-like locale facilities provided by the server operating system. For additional information refer to the documentation of your system.
7.1.1. Overview
Locale support is automatically initialized when a database cluster is created using initdb. initdb will initialize the database cluster with the locale setting of its execution environment; so if your system is already set to use the locale that you want in your database cluster then there is nothing else you need to do. If you want to use a different locale (or you are not sure which locale your system is set to), you can tell initdb exactly which locale you want with the option --locale. For example:
$ initdb --locale=sv_SE
This example sets the locale to Swedish (sv) as spoken in Sweden (SE). Other possibilities might be en_US (U.S. English) and fr_CA (Canada, French). If more than one character set can be useful for a locale then the specifications look like this: cs_CZ.ISO8859-2. What locales are available under what names on your system depends on what was provided by the operating system vendor and what was installed.
Occasionally it is useful to mix rules from several locales, e.g., use U.S. collation rules but Spanish messages. To support that, a set of locale subcategories exist that control only a certain aspect of the localization rules.
| LC_COLLATE | String sort order |
| LC_CTYPE | Character classification (What is a letter? The upper-case equivalent?) |
| LC_MESSAGES | Language of messages |
| LC_MONETARY | Formatting of currency amounts |
| LC_NUMERIC | Formatting of numbers |
| LC_TIME | Formatting of dates and times |
If you want the system to behave as if it had no locale support, use the special locale C or POSIX.
The nature of some locale categories is that their value has to be fixed for the lifetime of a database cluster. That is, once initdb has run, you cannot change them anymore. LC_COLLATE and LC_CTYPE are those categories. They affect the sort order of indexes, so they must be kept fixed, or indexes on text columns will become corrupt. PostgreSQL enforces this by recording the values of LC_COLLATE and LC_CTYPE that are seen by initdb. The server automatically adopts those two values when it is started.
The other locale categories can be changed as desired whenever the server is started by setting the run-time configuration variables that have the same name as the locale categories (see Section 3.4 for details). The defaults that are chosen by initdb are actually only written into the configuration file postgresql.conf to serve as defaults when the server is started. If you delete the assignments from postgresql.conf then the server will inherit the settings from the execution environment.
Note that the locale behavior of the server is determined by the environment variables seen by the server, not by the environment of any client. Therefore, be careful to configure the correct locale settings before starting the server. A consequence of this is that if client and server are set up to different locales, messages may appear in different languages depending on where they originated.
Note: When we speak of inheriting the locale from the execution environment, this means the following on most operating systems: For a given locale category, say the collation, the following environment variables are consulted in this order until one is found to be set: LC_ALL, LC_COLLATE (the variable corresponding to the respective category), LANG. If none of these environment variables are set then the locale defaults to C.
Some message localization libraries also look at the environment variable LANGUAGE which overrides all other locale settings for the purpose of setting the language of messages. If in doubt, please refer to the documentation of your operating system, in particular the gettext manual page, for more information.
To enable messages translated to the user's preferred language, the --enable-nls option must be used. This option is independent of the other locale support.
7.1.2. Benefits
Locale support influences in particular the following features:
The
to_charfamily of functionsThe LIKE and ~ operators for pattern matching
The only severe drawback of using the locale support in PostgreSQL is its speed. So use locale only if you actually need it. It should be noted in particular that selecting a non-C locale disables index optimizations for LIKE and ~ operators, which can make a huge difference in the speed of searches that use those operators.
7.1.3. Problems
If locale support doesn't work in spite of the explanation above, check that the locale support in your operating system is correctly configured. To check whether a given locale is installed and functional you can use Perl, for example. Perl has also support for locales and if a locale is broken perl -v will complain something like this:
$ export LC_CTYPE='not_exist'
$ perl -v
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LC_ALL = (unset),
LC_CTYPE = "not_exist",
LANG = (unset)
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
Check that your locale files are in the right location. Possible locations include: /usr/lib/locale (Linux, Solaris), /usr/share/locale (Linux), /usr/lib/nls/loc (DUX 4.0). Check the locale man page of your system if you are not sure.
Check that PostgreSQL is actually using the locale that you think it is. LC_COLLATE and LC_CTYPE settings are determined at initdb time and cannot be changed without repeating initdb. Other locale settings including LC_MESSAGES and LC_MONETARY are determined by the environment the postmaster is started in, and can be changed with a simple postmaster restart. You can check the LC_COLLATE and LC_CTYPE settings of a database with the contrib/pg_controldata utility program.
The directory src/test/locale contains a test suite for PostgreSQL's locale support.
Client applications that handle server-side errors by parsing the text of the error message will obviously have problems when the server's messages are in a different language. If you create such an application you need to devise a plan to cope with this situation. The embedded SQL interface (ecpg) is also affected by this problem. It is currently recommended that servers interfacing with ecpg applications be configured to send messages in English.
Maintaining catalogs of message translations requires the on-going efforts of many volunteers that want to see PostgreSQL speak their preferred language well. If messages in your language is currently not available or fully translated, your assistance would be appreciated. If you want to help, refer to the Developer's Guide or write to the developers' mailing list.