Re: RFC: Extension Packaging & Lookup - Mailing list pgsql-hackers

Thanks for this, David,

> On Oct 28, 2024, at 3:19 PM, David E. Wheeler <david@justatheory.com> wrote:
>
> ## Challenge: Third Party Dependencies

This of course is the area that worries the heck out of me, as someone with extensions that includes not just single
systemdependencies but long chains of them (depending on GDAL draws in a huge tree). 
>
> For solutions that require installing a DSO outside the default directories that the system is aware of, one needs a
wayto tell the system where to find them. There are two basic methods for doing so: 
>
> 1.  Set LD_LIBRARY_PATH to the directory in which third-party DSOs are installed. Today, however, this is considered
aninsecure pattern[7]. It doesn’t work at all on macOS, for example, unless you disable SIP[8]. Few will do so, nor
shouldthey. Andres Freund reports that it’s on its way out on Linux, too. So perhaps some can do this, but it sounds as
ifLD_LIBRARY_PATH’s days are numbered. 
>
> 2.  Use `-rpath` when compiling the DSOs to point to the proper place. This embeds the path in the DSO, so it always
looksin the same place. However, this requires that the DSO be recompiled for every variant of the `-rpath`, which
createschallenges for non-path specific binary packaging --- or if an OS vendor changes the director. But perhaps
Postgresitself could be compiled with an `-rpath` that will be used when loading extensions, so the extension DSOs
themselvesdon’t have to know the path? Then the base image just needs to be compiled with that option. 

I’m unsure if it will work, but I have wondered if building out the dependencies to install right next to the DSO, and
givingthe DSO an rpath of “.” would achieve the effect we are looking for. It’s unfortunate (DY)LD_LIBRARY_PATH is dead
anddying, but there we are. The trouble I see with somehow coercing the system to load a local copy of system libraries
isfor (a) common system libs that PostgreSQL itself might be linking (libssl, for example) that then will end up with
symbolcollisions between the copy loaded by postgres and the copy loaded by the DSO and (b) same thing but for
differentextensions with the same dependencies.  

I guess I cannot shake the idea that a lot of interesting extensions are going to have interesting system dependencies,
that“exposing an interesting library to postgres” has a high value for an integration system like PostgreSQL.  

Question for the more knowledgable, how are binary distribution systems like Conda and others shipping DLLs such that
differentpackages don’t clobber each other? 

P.

>
> ## Comments and Corrections
>
> I think I captured most of the issues we discussed at PGConf.eu <http://pgconf.eu/> last week; please correct any
misunderstandings,inaccuracies, and oversights you spot! And are there any other issues you can think of with the
overallapproach? 
>
> Thanks for reading to the end!
>
> Best,
>
> David
>
> [1]: https://www.postgresql.org/message-id/D30A91FA-A6D4-4737-941F-0BBB2984B730%40justatheory.com
> [2]: https://github.com/ossc-db/pg_hint_plan/
> [3]: https://commitfest.postgresql.org/50/4913/
> [4]: https://superuser.com/a/1603150/285886
> [5]: https://github.com/pramsey/pgsql-http
> [6]: https://stackoverflow.com/a/42260979/79202
> [7]: http://xahlee.info/UnixResource_dir/_/ldpath.html
> [8]: https://developer.apple.com/documentation/security/disabling-and-enabling-system-integrity-protection
>
>
>