Thread: pgsql: doc: mention dependency on collation libraries
doc: mention dependency on collation libraries Document that index storage is dependent on the operating system's collation library ordering, and any change in that ordering can create invalid indexes. Discussion: 20160617154311.GB19359@momjian.us Backpatch-through: 9.1 Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/b54f7a9ac9646845138f6851fdf3097e22daa383 Modified Files -------------- doc/src/sgml/runtime.sgml | 9 +++++++++ 1 file changed, 9 insertions(+)
Re: Bruce Momjian 2016-07-02 <E1bJMkm-0001gS-3c@gemulon.postgresql.org> > doc: mention dependency on collation libraries > > Document that index storage is dependent on the operating system's > collation library ordering, and any change in that ordering can create > invalid indexes. Shouldn't this mention that OS upgrades are a possible problem as well? We've seen the de_DE.UTF-8 ordering break in RHEL 5->6, and again in 6->7. (5 and 7 are compatible.) The problematic strings were "999" and "9-9-9". (German writeup: http://www.credativ.de/blog/postgresql-und-inkompatible-deutsche-spracheigenschaften-centosrhel) Christoph
On Sat, Jul 2, 2016 at 05:39:51PM +0200, Christoph Berg wrote: > Re: Bruce Momjian 2016-07-02 <E1bJMkm-0001gS-3c@gemulon.postgresql.org> > > doc: mention dependency on collation libraries > > > > Document that index storage is dependent on the operating system's > > collation library ordering, and any change in that ordering can create > > invalid indexes. > > Shouldn't this mention that OS upgrades are a possible problem as > well? We've seen the de_DE.UTF-8 ordering break in RHEL 5->6, and > again in 6->7. (5 and 7 are compatible.) The problematic strings were > "999" and "9-9-9". > > (German writeup: > http://www.credativ.de/blog/postgresql-und-inkompatible-deutsche-spracheigenschaften-centosrhel) Yes, we can do that. Do you have suggested wording? I was not sure how to tell people anything related to collation versions. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: Bruce Momjian 2016-07-02 <20160702155517.GD18610@momjian.us> > > Shouldn't this mention that OS upgrades are a possible problem as > > well? We've seen the de_DE.UTF-8 ordering break in RHEL 5->6, and > > again in 6->7. (5 and 7 are compatible.) The problematic strings were > > "999" and "9-9-9". > > Yes, we can do that. Do you have suggested wording? I was not sure how > to tell people anything related to collation versions. How about simply this: ? Non<literal>C</> and and non-<literal>POSIX</> locales rely on the operating system's collation library for character set ordering. This controls the ordering of keys stored in indexes. For this eason, a cluster cannot switch to an incompatible collation library ersion, either through + operating system upgrade, snapshot restore, binary streaming replication, or <application>pg_upgrade</> run. Christoph
On Sat, Jul 2, 2016 at 06:02:12PM +0200, Christoph Berg wrote: > Re: Bruce Momjian 2016-07-02 <20160702155517.GD18610@momjian.us> > > > Shouldn't this mention that OS upgrades are a possible problem as > > > well? We've seen the de_DE.UTF-8 ordering break in RHEL 5->6, and > > > again in 6->7. (5 and 7 are compatible.) The problematic strings were > > > "999" and "9-9-9". > > > > Yes, we can do that. Do you have suggested wording? I was not sure how > > to tell people anything related to collation versions. > > How about simply this: ? > > Non<literal>C</> and and non-<literal>POSIX</> locales rely on the > operating system's collation library for character set ordering. > This controls the ordering of keys stored in indexes. For this > eason, > a cluster cannot switch to an incompatible collation library > ersion, > either through > > + operating system upgrade, > > snapshot restore, binary streaming replication, or > <application>pg_upgrade</> run. OK, good point. I was more focused on cluster moves than an OS change. How is the attached patch? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
Re: Bruce Momjian 2016-07-02 <20160702172248.GE18610@momjian.us> > OK, good point. I was more focused on cluster moves than an OS change. > How is the attached patch? I like it. Christoph
On 7/2/16 11:22 AM, Bruce Momjian wrote: > doc: mention dependency on collation libraries > > Document that index storage is dependent on the operating system's > collation library ordering, and any change in that ordering can create > invalid indexes. I think you're missing some punctuation between "Non" and "C". -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sat, Jul 2, 2016 at 01:22:48PM -0400, Bruce Momjian wrote: > On Sat, Jul 2, 2016 at 06:02:12PM +0200, Christoph Berg wrote: > > Re: Bruce Momjian 2016-07-02 <20160702155517.GD18610@momjian.us> > > > > Shouldn't this mention that OS upgrades are a possible problem as > > > > well? We've seen the de_DE.UTF-8 ordering break in RHEL 5->6, and > > > > again in 6->7. (5 and 7 are compatible.) The problematic strings were > > > > "999" and "9-9-9". > > > > > > Yes, we can do that. Do you have suggested wording? I was not sure how > > > to tell people anything related to collation versions. > > > > How about simply this: ? > > > > Non<literal>C</> and and non-<literal>POSIX</> locales rely on the > > operating system's collation library for character set ordering. > > This controls the ordering of keys stored in indexes. For this > > eason, > > a cluster cannot switch to an incompatible collation library > > ersion, > > either through > > > > + operating system upgrade, > > > > snapshot restore, binary streaming replication, or > > <application>pg_upgrade</> run. > > OK, good point. I was more focused on cluster moves than an OS change. > How is the attached patch? With the mention of different operating systems, there is no need to mention pg_upgrade anymore as it is already covered. Updated patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Attachment
Re: Bruce Momjian 2016-07-20 <20160720161318.GE24559@momjian.us> > > > How about simply this: ? > > > > > > Non<literal>C</> and and non-<literal>POSIX</> locales rely on the > > > operating system's collation library for character set ordering. > > > This controls the ordering of keys stored in indexes. For this > > > eason, > > > a cluster cannot switch to an incompatible collation library > > > ersion, > > > either through > > > > > > + operating system upgrade, > > > > > > snapshot restore, binary streaming replication, or > > > <application>pg_upgrade</> run. > > > > OK, good point. I was more focused on cluster moves than an OS change. > > How is the attached patch? > > With the mention of different operating systems, there is no need to > mention pg_upgrade anymore as it is already covered. Updated patch > attached. With that argument, you could drop snapshot restore and replication as well, as "different OS (version)" is the real problem. Still, mentioning pg_upgrade would highlight the fact that it doesn't "fix" the problem in the same sense that pg_dump would. Collation changes are subtle changes that are surprising in many cases, so mention a few cases makes people aware more. Christoph
On Thu, Jul 21, 2016 at 09:59:42AM +0200, Christoph Berg wrote: > Re: Bruce Momjian 2016-07-20 <20160720161318.GE24559@momjian.us> > > > > How about simply this: ? > > > > > > > > Non<literal>C</> and and non-<literal>POSIX</> locales rely on the > > > > operating system's collation library for character set ordering. > > > > This controls the ordering of keys stored in indexes. For this > > > > eason, > > > > a cluster cannot switch to an incompatible collation library > > > > ersion, > > > > either through > > > > > > > > + operating system upgrade, > > > > > > > > snapshot restore, binary streaming replication, or > > > > <application>pg_upgrade</> run. > > > > > > OK, good point. I was more focused on cluster moves than an OS change. > > > How is the attached patch? > > > > With the mention of different operating systems, there is no need to > > mention pg_upgrade anymore as it is already covered. Updated patch > > attached. > > With that argument, you could drop snapshot restore and replication as > well, as "different OS (version)" is the real problem. Still, > mentioning pg_upgrade would highlight the fact that it doesn't "fix" > the problem in the same sense that pg_dump would. Collation changes > are subtle changes that are surprising in many cases, so mention a few > cases makes people aware more. I mentioned snapshot restore and replication as a way to highlight that it is binary transfer that is the problem, not pg_dump. You are right about pg_upgrade not fixing the problem, but since it only runs on the same machine, I don't see it as a good example. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Sat, Jul 9, 2016 at 09:17:42AM -0400, Peter Eisentraut wrote:
> On 7/2/16 11:22 AM, Bruce Momjian wrote:
> >doc: mention dependency on collation libraries
> >
> >Document that index storage is dependent on the operating system's
> >collation library ordering, and any change in that ordering can create
> >invalid indexes.
>
> I think you're missing some punctuation between "Non" and "C".
Sorry I am just getting to this; it has been already fixed: :-)
commit 42ec6c2da699e8e0b1774988fa97297a2cdf716c
Author: Stephen Frost <sfrost@snowman.net>
Date: Wed Jul 13 09:17:35 2016 -0400
Add missing hyphen
Pointed out by Alexander Law
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Re: Bruce Momjian 2016-07-26 <20160726213526.GA27321@momjian.us> > I mentioned snapshot restore and replication as a way to highlight that > it is binary transfer that is the problem, not pg_dump. > > You are right about pg_upgrade not fixing the problem, but since it only > runs on the same machine, I don't see it as a good example. Ok, go for it! Christoph
On Mon, Aug 1, 2016 at 01:04:30PM +0200, Christoph Berg wrote: > Re: Bruce Momjian 2016-07-26 <20160726213526.GA27321@momjian.us> > > I mentioned snapshot restore and replication as a way to highlight that > > it is binary transfer that is the problem, not pg_dump. > > > > You are right about pg_upgrade not fixing the problem, but since it only > > runs on the same machine, I don't see it as a good example. > > Ok, go for it! Applied and backpatched to 9.1. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +