On 17/06/2025 20:14, Jeff Davis wrote:
> On Tue, 2025-06-17 at 17:37 +0200, Vik Fearing wrote:
>> If the character set of <character factor> is UTF8, UTF16, or UTF32,
>> then FR is replaced by
>> Case:
>> i) If the <search condition> S IS NORMALIZED evaluates to
>> True, then NORMALIZE (FR)
>> ii) Otherwise, FR.
> I read that as "if the input is normalized, then the output should be
> normalized", IOW preserve the normalization. But does it mean "preserve
> whatever the input normal form is" or "preserve NFC if the input is
> NFC, otherwise the normalization is undefined"?
>
> The above wording seems to mean "preserve NFC if the input is NFC",
> because that's what NORMALIZE(FR) does when the normal form is
> unspecified.
Yes, and that is also the default for <normalized predicate>.
>> It does not appear to me that our LOWER and UPPER functions obey this
>> rule,
> You are correct:
>
> WITH s(t) AS
> (SELECT NORMALIZE(U&'\00C1\00DF\0301' COLLATE "en-US-x-icu"))
> SELECT UPPER(t) = NORMALIZE(UPPER(t)) FROM s;
> ?column?
> ----------
> f
>
>> so there is a valid argument that we should continue to ignore it.
>> Or, we can say that we have at least one of three compliant.
> What do other databases do?
I don't know. I am just pointing out what the Standard says. I think
we should either comply, or say that we don't do it for LOWER and UPPER
so let's keep things implementation-consistent.
> Given how costly normalization can be, imposing that on every caller
> seems like a bit much.
How much does it cost to check for NFC? I honestly don't know the
answer to that question, but that is the only case where we need to
maintain normalization.
> And favoring NFC for the user unconditionally
> might not be the best thing. Then again, NFC is good most of the time,
> and there are patches to speed up normalization.
It's not unconditionally, it's only if the input was NFC.
> I tend to think that a lot of users who want casefolding would also
> want normalization, but it's hard to weigh that against the performance
> cost. It might not matter outside of a few edge cases, though I'm not
> sure exactly how many.
I defer to you and others in the thread to make this decision.
--
Vik Fearing
