Re: Direct I/O - Mailing list pgsql-hackers

Andrew Dunstan <andrew@dunslane.net> writes:

> On 2023-04-12 We 10:23, Dagfinn Ilmari Mannsåker wrote:
>> Andrew Dunstan<andrew@dunslane.net>  writes:
>>
>>> On 2023-04-12 We 01:48, Thomas Munro wrote:
>>>> On Wed, Apr 12, 2023 at 3:04 PM Thomas Munro<thomas.munro@gmail.com>   wrote:
>>>>> On Wed, Apr 12, 2023 at 2:56 PM Christoph Berg<myon@debian.org>   wrote:
>>>>>> I'm hitting a panic in t_004_io_direct. The build is running on
>>>>>> overlayfs on tmpfs/ext4 (upper/lower) which is probably a weird
>>>>>> combination but has worked well for building everything over the last
>>>>>> decade. On Debian unstable:
>>>>>>
>>>>>> PANIC:  could not open file "pg_wal/000000010000000000000001": Invalid argument
>>>>> ... I have a new idea:  perhaps it is possible to try
>>>>> to open a file with O_DIRECT from perl, and if it fails like that,
>>>>> skip the test.  Looking into that now.
>>>> I think I have that working OK.  Any Perl hackers want to comment on
>>>> my use of IO::File (copied from examples on the internet that showed
>>>> how to use O_DIRECT)?  I am not much of a perl hacker but according to
>>>> my package manager, IO/File.pm came with perl itself.  And the Fcntl
>>>> eval trick that I copied from File::stat, and the perl-critic
>>>> suppression that requires?
>>>
>>> I think you can probably replace a lot of the magic here by simply saying
>>>
>>>
>>> if (Fcntl->can("O_DIRECT")) ...
>> Fcntl->can() is true for all constants that Fcntl knows about, whether
>> or not they are defined for your OS. `defined &O_DIRECT` is the simplest
>> check, see my other reply to Thomas.
>>
>>
>
> My understanding was that Fcntl only exported constants known to the
> OS. That's certainly what its docco suggests, e.g.:
>
>     By default your system's F_* and O_* constants (eg, F_DUPFD and
> O_CREAT)
>     and the FD_CLOEXEC constant are exported into your namespace.

It's a bit more magical than that (this is Perl after all).  They are
all exported (which implicitly creates stubs visible to `->can()`,
similarly to forward declarations like `sub O_FOO;`), but only the
defined ones (`#ifdef O_FOO` is true) are defined (`defined &O_FOO` is
true).  The rest fall through to an AUTOLOAD¹ function that throws an
exception for undefined ones.

Here's an example (Fcntl knows O_RAW, but Linux does not define it):

    $ perl -E '
        use strict; use Fcntl;
        say "can", main->can("O_RAW") ? "" : "not";
        say defined &O_RAW ? "" : "not ", "defined";
        say O_RAW;'
    can
    not defined
    Your vendor has not defined Fcntl macro O_RAW, used at -e line 4

While O_DIRECT is defined:

    $ perl -E '
        use strict; use Fcntl;
        say "can", main->can("O_DIRECT") ? "" : "not";
        say defined &O_DIRECT ? "" : "not ", "defined";
        say O_DIRECT;'
    can
    defined
    16384

And O_FOO is unknown to Fcntl (the parens on `O_FOO()q are to make it
not a bareword, which would be a compile error under `use strict;` when
the sub doesn't exist at all):

    $ perl -E '
        use strict; use Fcntl;
        say "can", main->can("O_FOO") ? "" : "not";
        say defined &O_FOO ? "" : "not ", "defined";
        say O_FOO();'
    cannot
    not defined
    Undefined subroutine &main::O_FOO called at -e line 4.

> cheers
>
>
> andrew

- ilmari

[1] https://perldoc.perl.org/perlsub#Autoloading