Thread: client encoding that psql command sets
Hi all,
When I tried to psql command, I found that the result of command seems to be
different from explanation of the manual. Please tell me which is
correct performance, the result of command or the explanation of manual.
I show the details as follows.
[Events］
The encoding that psql command sets has difference between the result of command
and the explanation of manual.
The following is the explanation of encoding that psql command sets,
which I made extracts from the 9.2 manual.
---------------------------------------------------------------------------
VI. Reference / II. PostgreSQL Client Applications / psql / Connecting to a Database
"If at least one of standard input or standard output are a terminal,
then psql sets the client encoding to "auto",
which will detect the appropriate client encoding from the locale settings
(LC_CTYPE environment variable on Unix systems).
If this doesn't work out as expected,
the client encoding can be overridden using the environment variable PGCLIENTENCODING."
----------------------------------------------------------------------------
Above explanation says that "If at least one of standard input or standard output
are a terminal, then psql sets the client encoding to "auto",
which will detect the appropriate client encoding from the locale settings
(LC_CTYPE environment variable on Unix systems)".
But according to the result from psql command, it seems that the appropriate
client encoding is NOT detected from the locale settings.
The result of psql command is following.
[Precondition of test]
- PostgreSQL VL：9.2.4
- Locale settings encoding : ja_JP.eucJP (LANG)
- Client encoding：UTF8
- client_encoding(paramater of postgresql.conf) : None specified
- "test.txt" is psql's input file which is written "\encoding".
[Result]
1) In case both standard input and standard output are a terminal
$ psql postgres -f test.txt -o result.txt
EUC_JP
→ Locale setting encoding
2) In case only standard input is a terminal
$ psql postgres -f test.txt > result.txt
$ cat result.txt
UTF8
→Client encoding
3) In case only standard output is a terminal
$ psql postgres -o result.txt < test.txt
UTF8
→Client encoding
4) In case both standard input and standard output are not a terminal
$ psql postgres < test.txt > result.txt
$ cat result.txt
UTF8
→Client encoding
1) and 4) performed just as manual, but 2) and 3) do not.
If 2) and 3) performed just as munual, I think that their encoding should be EUC_JP
that is locale setting encoding.
If manual has mistake, I think that correct explanation is following.
"If BOTH standard input AND standard output are a terminal, then psql sets
the client encoding to "auto", which will detect the appropriate client encoding
from the locale settings (LC_CTYPE environment variable on Unix systems)."
What do you think?
Warmest Regards.
On 02/05/2014 11:43 PM, 坂本 翼 wrote: > Hi all, > > When I tried to psql command, I found that the result of command seems to be > different from explanation of the manual. Please tell me which is > correct performance, the result of command or the explanation of manual. > I show the details as follows. > > [Events］ > The encoding that psql command sets has difference between the result of command > and the explanation of manual. > > The following is the explanation of encoding that psql command sets, > which I made extracts from the 9.2 manual. > --------------------------------------------------------------------------- > VI. Reference / II. PostgreSQL Client Applications / psql / Connecting to a Database > "If at least one of standard input or standard output are a terminal, > then psql sets the client encoding to "auto", > which will detect the appropriate client encoding from the locale settings > (LC_CTYPE environment variable on Unix systems). > If this doesn't work out as expected, > the client encoding can be overridden using the environment variable PGCLIENTENCODING." > ---------------------------------------------------------------------------- > Above explanation says that "If at least one of standard input or standard output > are a terminal, then psql sets the client encoding to "auto", > which will detect the appropriate client encoding from the locale settings > (LC_CTYPE environment variable on Unix systems)". > But according to the result from psql command, it seems that the appropriate > client encoding is NOT detected from the locale settings. > > The result of psql command is following. > > [Precondition of test] > - PostgreSQL VL：9.2.4 > - Locale settings encoding : ja_JP.eucJP (LANG) Not sure that it makes a difference but the docs say psql looks at LC_CTYPE not LANG for Unix systems. You did not say what OS you are working on though from the examples I am guessing some form of Unix. > - Client encoding：UTF8 > - client_encoding(paramater of postgresql.conf) : None specified > - "test.txt" is psql's input file which is written "\encoding". > > [Result] > 1) In case both standard input and standard output are a terminal > $ psql postgres -f test.txt -o result.txt > EUC_JP > → Locale setting encoding > > 2) In case only standard input is a terminal > $ psql postgres -f test.txt > result.txt > $ cat result.txt > UTF8 > →Client encoding > > 3) In case only standard output is a terminal > $ psql postgres -o result.txt < test.txt > UTF8 > →Client encoding > > 4) In case both standard input and standard output are not a terminal > $ psql postgres < test.txt > result.txt > $ cat result.txt > UTF8 > →Client encoding > > 1) and 4) performed just as manual, but 2) and 3) do not. > If 2) and 3) performed just as munual, I think that their encoding should be EUC_JP > that is locale setting encoding. > > If manual has mistake, I think that correct explanation is following. > > "If BOTH standard input AND standard output are a terminal, then psql sets > the client encoding to "auto", which will detect the appropriate client encoding > from the locale settings (LC_CTYPE environment variable on Unix systems)." > > What do you think? > > Warmest Regards. > > > > > > > -- Adrian Klaver adrian.klaver@gmail.com
> Not sure that it makes a difference but the docs say psql looks at > LC_CTYPE not LANG for Unix systems. You did not say what OS you are > working on though from the examples I am guessing some form of Unix. Thank you for the response. Sorry, I had not indicated OS information. OS information is the following: Red Hat Enterprise Linux Server release 6.2 Kernel 2.6.32-220.el6.x86_64 on an x86_64 The reason which was being verified using the LANG environment variable, I thought that the value of a LANG environment variablewas set as LC_CTYPE when the LC_CTYPE environment variable and the LC_ALL environment variable are not set up. The LC_CTYPE environment variable was set up and re-verified. The result of psql command is following. [Result] % setenv LC_CTYPE ja_JP.eucJP % psql postgres -f test.txt -o result.txt EUC_JP % psql postgres -f test.txt > result.txt UTF8 % psql postgres -o result.txt < test.txt UTF8 % psql postgres < test.txt > result.txt UTF8 Even when a LC_CTYPE environment variable was set up, the result did not change. What do you think?
[CC'ed -hackers]
Tsubasa Sakamoto wrote:
>> Not sure that it makes a difference but the docs say psql looks at
>> LC_CTYPE not LANG for Unix systems. You did not say what OS you are
>> working on though from the examples I am guessing some form of Unix.
> The LC_CTYPE environment variable was set up and re-verified.
> The result of psql command is following.
>
> [Result]
> % setenv LC_CTYPE ja_JP.eucJP
>
> % psql postgres -f test.txt -o result.txt EUC_JP
>
> % psql postgres -f test.txt > result.txt
> UTF8
>
> % psql postgres -o result.txt < test.txt
> UTF8
>
> % psql postgres < test.txt > result.txt
> UTF8
>
> Even when a LC_CTYPE environment variable was set up, the result did not change.
> What do you think?
I think that the documentation contradicts the code.
In bin/psql/settings.h:
typedef struct _psqlSettings
{
[...]
bool notty; /* stdin or stdout is not a tty (as determined
* on startup) */
[...]
} PsqlSettings;
extern PsqlSettings pset;
In bin/psql/command.c and bin/psql/startup.c:
keywords[6] = "client_encoding";
values[6] = (pset.notty || getenv("PGCLIENTENCODING")) ? NULL : "auto";
That matches your observations: client_encoding=auto is only
used if both stdin and stdout are attached to a tty.
I suggest the attached documentation fix.
Yours,
Laurenz Albe
Attachment
Thanks for your support.
Best regards,
Tsubasa Sakamoto
> -----Original Message-----
> From: Albe Laurenz [mailto:laurenz.albe@wien.gv.at]
> Sent: Friday, February 07, 2014 6:12 PM
> To: Tsubasa Sakamoto *EXTERN*; 'Adrian Klaver'
> Cc: pgsql-general@postgresql.org; pgsql-hackers@postgresql.org
> Subject: RE: [GENERAL] client encoding that psql command sets
>
> [CC'ed -hackers]
>
> Tsubasa Sakamoto wrote:
> >> Not sure that it makes a difference but the docs say psql looks at
> >> LC_CTYPE not LANG for Unix systems. You did not say what OS you are
> >> working on though from the examples I am guessing some form of Unix.
>
> > The LC_CTYPE environment variable was set up and re-verified.
> > The result of psql command is following.
> >
> > [Result]
> > % setenv LC_CTYPE ja_JP.eucJP
> >
> > % psql postgres -f test.txt -o result.txt EUC_JP
> >
> > % psql postgres -f test.txt > result.txt
> > UTF8
> >
> > % psql postgres -o result.txt < test.txt
> > UTF8
> >
> > % psql postgres < test.txt > result.txt
> > UTF8
> >
> > Even when a LC_CTYPE environment variable was set up, the result did not
> change.
> > What do you think?
>
> I think that the documentation contradicts the code.
>
> In bin/psql/settings.h:
>
> typedef struct _psqlSettings
> {
> [...]
> bool notty; /* stdin or stdout is not a tty (as
> determined
> * on startup) */ [...] } PsqlSettings;
>
> extern PsqlSettings pset;
>
> In bin/psql/command.c and bin/psql/startup.c:
>
> keywords[6] = "client_encoding";
> values[6] = (pset.notty || getenv("PGCLIENTENCODING")) ? NULL :
> "auto";
>
> That matches your observations: client_encoding=auto is only used if both
> stdin and stdout are attached to a tty.
>
> I suggest the attached documentation fix.
>
> Yours,
> Laurenz Albe
On Fri, Feb 7, 2014 at 09:12:10AM +0000, Albe Laurenz wrote:
> > Even when a LC_CTYPE environment variable was set up, the result did not change.
> > What do you think?
>
> I think that the documentation contradicts the code.
>
> In bin/psql/settings.h:
>
> typedef struct _psqlSettings
> {
> [...]
> bool notty; /* stdin or stdout is not a tty (as determined
> * on startup) */
> [...]
> } PsqlSettings;
>
> extern PsqlSettings pset;
>
> In bin/psql/command.c and bin/psql/startup.c:
>
> keywords[6] = "client_encoding";
> values[6] = (pset.notty || getenv("PGCLIENTENCODING")) ? NULL : "auto";
>
> That matches your observations: client_encoding=auto is only
> used if both stdin and stdout are attached to a tty.
>
> I suggest the attached documentation fix.
Patch applied and backpatched to 9.3. Thanks.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +