Re: Permute underscore separated components of columns before fuzzy matching - Mailing list pgsql-hackers

Hi!

Mikhail Gribkov <youzhick(at)gmail(dot)com> writes:

 > > Honestly I'm not entirely sure fixing only two switched words is 
worth the
 > > effort, but the declared goal is clearly achieved.
 >
 >
 > > I think the patch is good to go, although you need to fix code 
formatting.
 >
 >
 > I took a brief look at this.  I concur that we shouldn't need to be
 > hugely concerned about the speed of this code path.  However, we *do*
 > need to be concerned about its maintainability, and I think the patch
 > falls down badly there: it adds a chunk of very opaque and essentially
 > undocumented code, that people will need to reverse-engineer anytime
 > they are studying this function.  That could be alleviated perhaps
 > with more work on comments, but I have to wonder whether it's worth
 > carrying this logic at all.  It's a rather strange behavior to add,
 > and I wonder if many users will want it.

I encounter this problem all the time. I don't know, whether my clients 
are representative. But I see the problem, when the developers show me 
their code base all the time.
It's an issue for column names and table names alike. I personally spent 
hours watching developers trying various permutations.
They rarely request this feature. Usually they are to embarrassed for 
not knowing their object names to request anything in that state.
But I want the database, which I support, to be gentle and helpful to 
the user under these circumstances.

Regarding complexity: I think the permutation matrix is the thing to 
easily get wrong. I had a one off bug writing it down initially.
I tried to explain the conceptual approach better with a longer comment 
than before.

                 /*
                  * Only consider mirroring permutations, since the 
three simple rotations are already
                  * (or will be for a later underscore_current) covered 
above.
                  *
                  * The entries of the permutation matrix tell us, where 
we should copy the tree segments to.
                  * The zeroth dimension iterates over the permutations, 
while the first dimension iterates
                  * over the three segments are permuted to.
                  * Considering the string A_B_C the three segments are:
                  * - before the initial underscore sections (A)
                  * - between the underscore sections (B)
                  * - after the later underscore sections (C)
                  */

If anything is still unclear, I'd appreciate feedback about what might 
be still unclear/confusing about this.
I can't promise to be helpful, if something breaks. But I have 
practically forgotten how I did it, and I found it easy to extend it 
like described below. It would have been embarrassing otherwise. Yet 
this gives me hope, it should be possible to enable others the same way.
I certainly want the code simple without need to reverse-engineer 
anything. Please let me know, if there are difficult to understand bits 
left around.

 > One thing that struck me is that no care is being taken for adjacent
 > underscores (that is, "foo__bar" and similar cases).  It seems
 > unlikely that treating the zero-length substring between the
 > underscores as a word to permute is helpful; moreover, it adds
 > an edge case that the string-moving logic could easily get wrong.
 > I wonder if the code should treat any number of consecutive
 > underscores as a single separator.  (Somewhat related: I think it
 > will behave oddly when the first or last character is '_', since the
 > outer loop ignores those positions.)

I wasn't sure how there could be any potential future bug with copying 
zero-length strings, i.e. doing nothing. And I still don't see that.

There is one point I agree with: Doing this seems rarely helpful. I 
changed the code, so it treats sections delimited by an arbitrary amount 
of underscores.
So it never permutes with zero length strings within. I also added 
functionality to skip the zero length cases if we should encounter them 
at the end of the string.
So afaict there should be no zero length swaps left. Please let me know 
whether this is more to your liking.

I also replaced the hard limit of underscores with more nuanced limits 
of permutations to try before giving up.

 > > And it would be much more convenient to work with your patch if 
every next
 > > version file will have a unique name (maybe something like "_v2", "_v3"
 > > etc. suffixes)
 >
 >
 > Please.  It's very confusing when there are multiple identically-named
 > patches in a thread.

Sorry, I started with this, because I confused cf bot in the past about 
whether the patches should be applied on top of each other or not.

For me the cf-bot logic is a bit opaque there. But you are right, 
confusing patch readers is definitely worse. I'll try to do that. I hope 
the attached format is better.


One question about pgindent: I struggled a bit with getting the right 
version of bsd_indent. I found versions labeled 2.2.1 and 2.1.1, but 
apparently we work with 2.1.2. Where can I get that?

Regards
Arne