Re: pg_preadv() and pg_pwritev() - Mailing list pgsql-hackers

On 15.01.2021 01:13, Tom Lane wrote:
> I borrowed my wife's Mac, which is still on Catalina and up to now
> never had Xcode on it, and found some very interesting things.
> 
> Step 1: download/install Xcode 12.3, open it, agree to license,
> wait for it to finish "installing components".
> 
> At this point, /Library/Developer/CommandLineTools doesn't exist,
> and we have the following outputs from various probe commands:
> 
> % xcrun --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
> % xcrun --sdk macosx --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> % xcodebuild -version -sdk macosx Path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> 
> Also, cc -v reports
>    -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
> 
> Unsurprisingly, Xcode 12.3 itself only contains
> 
> % ls -l /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
> total 0
> drwxr-xr-x  5 root  wheel  160 Nov 30 07:27 DriverKit20.2.sdk
> drwxr-xr-x  7 root  wheel  224 Nov 30 07:27 MacOSX.sdk
> lrwxr-xr-x  1 root  wheel   10 Jan 14 15:57 MacOSX11.1.sdk -> MacOSX.sdk
> 
> Step 2: install command line tools (I used "xcode-select --install"
> to fire this off, rather than the Xcode menu item).
> 
> Now I have
> 
> % ls -l /Library/Developer/CommandLineTools/SDKs
> total 0
> lrwxr-xr-x  1 root  wheel   14 Jan 14 16:42 MacOSX.sdk -> MacOSX11.1.sdk
> drwxr-xr-x  8 root  wheel  256 Jul  9  2020 MacOSX10.15.sdk
> drwxr-xr-x  7 root  wheel  224 Nov 30 07:33 MacOSX11.1.sdk
> 
> which is pretty interesting in itself, because the same directory on
> my recently-updated-to-Big-Sur Macs does NOT have the 11.1 SDK.
> I wonder what determines which versions get installed here.
> 
> More interesting yet:
> 
> % xcrun --show-sdk-path
> /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
> % xcrun --sdk macosx --show-sdk-path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> % xcodebuild -version -sdk macosx Path
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
> 
> and cc -v reports
>    -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
> 
> So apparently, "xcrun --show-sdk-path" (without any -sdk option)
> is the most authoritative guide to the compiler's default sysroot.
> 
> However, googling turns up various people reporting that "xcrun
> --show-sdk-path" returns an empty string for them, and our last
> major investigation into this [1] found that there are some system
> states where the compiler appears to have no default sysroot,
> which I bet is the same thing.  I do not at this point have a recipe
> to reproduce such a state, but we'd be fools to imagine it's no
> longer possible.  My guess about it is that Apple's processes for
> updating the default sysroot during system updates are just plain
> buggy, with various corner cases that have ill-understood causes.
> 
> Also, after re-reading [1] I am not at all excited about trying to
> remove the -isysroot switches from our *FLAGS.  What I propose to do
> is keep that, but improve our mechanism for choosing a default value
> for PG_SYSROOT.  It looks like first trying "xcrun --show-sdk-path",
> and falling back to "xcodebuild -version -sdk macosx Path" if that
> doesn't yield a valid path, is more likely to give a working build
> than relying entirely on xcodebuild.  Maybe there's a case for trying
> "xcrun --sdk macosx --show-sdk-path" in between; in my tests that
> seemed noticeably faster than invoking xcodebuild, and I've not yet
> seen a case where it gave a different answer.
> 
> Thoughts?
> 
>             regards, tom lane
> 
> [1] https://www.postgresql.org/message-id/flat/20840.1537850987%40sss.pgh.pa.us
> 

Thanks for thorough investigation and sorry for the late reply.

I spent quite some time trying to understand / reverse engineer the 
logic behind xcrun's default SDK selection. Apparently, "man xcrun" is 
not accurate saying:

    The  SDK  which  will  be searched defaults to the most recent 
available...

I didn't find anything really useful or helpful. 
"/Library/Developer/CommandLineTools" is hardcoded into 
"libxcrun.dylib". On my machine xcrun scans

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs

and

/Library/Developer/CommandLineTools/SDKs

in that order, and loads "SDKSettings.plist" from each subdirectory. I 
looked into plists, but couldn't find anything special about 
"MacOSX10.15.sdk".


Okay, here is what I have:

% ls -l 
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
total 0
drwxr-xr-x  5 root  wheel  160 Nov 30 15:27 DriverKit20.2.sdk
drwxr-xr-x  7 root  wheel  224 Nov 30 15:27 MacOSX.sdk
lrwxr-xr-x  1 root  wheel   10 Dec 17 14:25 MacOSX11.1.sdk -> MacOSX.sdk

% ls -l /Library/Developer/CommandLineTools/SDKs
total 0
lrwxr-xr-x  1 root  wheel   14 Nov 17 02:21 MacOSX.sdk -> MacOSX11.0.sdk
drwxr-xr-x  8 root  wheel  256 Nov 17 02:22 MacOSX10.15.sdk
drwxr-xr-x  7 root  wheel  224 Oct 19 23:39 MacOSX11.0.sdk

% xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk

% xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk

Oh, that's weird! Nevertheless I like you suggestion to call "xcrun" 
from "configure".

Adding "--verbose" doesn't really explain anything, but just in case.

% xcrun --verbose --no-cache --find cc
xcrun: note: PATH = 
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT = 
'/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db = 
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: xcrun via cc (xcrun)
xcrun: note: database key is: 
cc|/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk||/Applications/Xcode.app/Contents/Developer|
xcrun: note: looking up with 
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk 
macosx -find cc 2> /dev/null'
xcrun: note: lookup resolved with 'xcodebuild -find' to 
'/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc


% xcrun --verbose --no-cache --sdk macosx --find cc
xcrun: note: looking up SDK with 
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk 
macosx -version Path'
xcrun: note: PATH = 
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT = 'macosx'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db = 
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: lookup resolved to: 
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: looking up SDK with 
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk 
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk 
-version PlatformPath'
xcrun: note: PATH = 
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT = 
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db = 
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: lookup resolved to:
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform'
xcrun: note: PATH = 
'/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin'
xcrun: note: SDKROOT = 
'/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk'
xcrun: note: TOOLCHAINS = ''
xcrun: note: DEVELOPER_DIR = '/Applications/Xcode.app/Contents/Developer'
xcrun: note: XCODE_DEVELOPER_USR_PATH = ''
xcrun: note: xcrun_db = 
'/var/folders/8x/jvqv7hyd5h98m7tz2zm9r0yc0000gn/T/xcrun_db'
xcrun: note: xcrun via cc (xcrun)
xcrun: note: database key is: 

cc|/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk||/Applications/Xcode.app/Contents/Developer|
xcrun: note: looking up with 
'/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk 
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk 
-find cc 2> /dev/null'
xcrun: note: lookup resolved with 'xcodebuild -find' to 
'/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc