Fwd: [BUGS] pg_trgm word_similarity inconsistencies or bug - Mailing list pgsql-bugs

Hello Alexander,

We agree that the current pg_trgm documentation does not correctly reflect the de facto behavior of word_similarity(), and that something has to be changed. But to us, it is more a documentation problem than anything else.

What is computed  is still « substring_similarity » as was initially specified between us,  but it is influenced by a strong word boundary bias  caused by the way trigrams are padded at word boundaries. This bias was noticed by early reviewers and you explained that this motivated the name switch to « word_similarity ». As you will remember, at the time we discovered this, we were suprised because we considerd this as a slight misnomer.  Indeed, what is  currently described in the 9.6 pg_trgm documentation is inaccurate (although seemingly consistent with this new name)  and has to be amended.

Now, word_similarity() has been out for more than a year and, of course, it is preferable to avoid any breaking changes… In our case, we consider the name « unfortunate » and the explanation buggy, not the function itself.

As you may remember from the initial discussion, some other users stressed the importance to be able to matchsub strings. We tend to agree with what Jeff Janes wrote in this discussion :

Now, considering your proposal :

As far as we are concerned, we use <% and %>  everyday  for efficient fuzzy matching on large databases. Our typical usage scenario is matching noisy OCRized text strings against reference databases.

> 1) Define GUC variable which specifies whether word_similarity() should force extent boundaries to be at word boundaries,

Ok for us, iff default behavior remains the same as now, for backward compatibility reasons. We could take advantage, in some cases, of the new « word rounded » behavior controlled by the GUC variable, but this would not cover all scenarios currently in use.

> 2) Document both cases of word_similarity() behavior.

This is clearly needed anyway.

Best regards,

François CHAHUNEAU Directeur des technologies

NUMEN DIGITAL| 24, rue Marc Seguin 75018 Paris France | www.numen.fr Tel +33 1 40 37 95 03 | Mob +33 6 07 85 21 79 | Fax +33 1 40 37 94 94

Pensez vert, n’imprimez que nécessaire. Les informations contenues dans le présent e-mail sont exclusivement adressées au(x) destinataire(s) de ce message et peuvent contenir des informations confidentielles, protégées par un secret professionnel. L’utilisation de ces informations par d’autres personnes que le(s) destinataire(s) est strictement interdite. Si vous n’êtes pas destinataire de ce message, la publication, la reproduction, la diffusion et /ou la distribution de ces informations auprès de tiers n’est pas autorisée. Si vous avez reçu cet e-mail par erreur, veuillez nous en informer immédiatement, détruire l'email, ses copies et documents joints et le supprimer.