Thread: Compilation of timezone source with zic fails (on mountpoint)

Hi,

During building of the ongoing PG updates (9.6.1, 9.5.5 to 9.1.24), we
found that the build failed for Linux on 9.3 and 9.2 with the following
error:
--
/bin/mkdir -p '/mnt/hgfs/pginstaller-repo/server/staging/linux/share/
postgresql'
./zic -d '/mnt/hgfs/pginstaller-repo/server/staging/linux/share/postgresql/timezone'
-p 'US/Eastern' ./data/africa ./data/antarctica ./data/asia
./data/australasia ./data/europe ./data/northamerica ./data/southamerica
./data/pacificnew ./data/etcetera ./data/factory ./data/backward
./data/systemv
warning: "./data/africa", line 159: symbolic link used because hard link
failed: Operation not permitted
warning: "./data/africa", line 160: symbolic link used because hard link
failed: Operation not permitted
....
....
warning: "./data/backward", line 125: symbolic link used because hard link
failed: Operation not permitted
.*/zic: link from
/mnt/hgfs/pginstaller-repo/**server/staging/linux/share/postgresql/timezone/US/Eastern
failed: Operation not permitted*
make[2]: *** [install] Error 1
--

On investigating, we found that it was because of the few commits in the
new updates related to zic.c which tries to create hard links. These hard
links failed to create on our build farm because we were building the
source on the mount point (/mnt/hgfs/) where hard links cannot be created.
The Linux machine is a Virtual Machine hosted on VMware Fusion on the Mac
Server and the Fusion doesn't allow to create hard links inside the shared
location. It errors out with "Operation not permitted". Symbolic links can
be created though.

On other build farm where we don't have VMs installed on Fusion, this error
is not seen.

--


--
Sandeep Thakkar

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> On investigating, we found that it was because of the few commits in the
> new updates related to zic.c which tries to create hard links. These hard
> links failed to create on our build farm because we were building the
> source on the mount point (/mnt/hgfs/) where hard links cannot be created.
> The Linux machine is a Virtual Machine hosted on VMware Fusion on the Mac
> Server and the Fusion doesn't allow to create hard links inside the shared
> location. It errors out with "Operation not permitted". Symbolic links can
> be created though.

Although the IANA guys did fool around with their link-creation code in
the latest update, zic has always attempted to create hard links between
timezone files.  How did that build infrastructure ever work?

I'm not especially inclined to regard this as a bug...

            regards, tom lane

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> During building of the ongoing PG updates (9.6.1, 9.5.5 to 9.1.24), we
> found that the build failed for Linux on 9.3 and 9.2 with the following
> error:
> ...
> warning: "./data/backward", line 125: symbolic link used because hard li=
nk
> failed: Operation not permitted
> .*/zic: link from
> /mnt/hgfs/pginstaller-repo/**server/staging/linux/share/postgresql/timez=
one/US/Eastern
> failed: Operation not permitted*
> make[2]: *** [install] Error 1

> On investigating, we found that it was because of the few commits in the
> new updates related to zic.c which tries to create hard links. These har=
d
> links failed to create on our build farm because we were building the
> source on the mount point (/mnt/hgfs/) where hard links cannot be create=
d.
> The Linux machine is a Virtual Machine hosted on VMware Fusion on the Ma=
c
> Server and the Fusion doesn't allow to create hard links inside the shar=
ed
> location. It errors out with "Operation not permitted". Symbolic links c=
an
> be created though.

I took a closer look at this.  As I said earlier, zic has always attempted
to use hard links, and now that I look, the fallback to symlinks has been
there right along as well.  So you were always getting those warnings on
that platform.  What is new, apparently, is this bit at the end of zic.c's
main() which came in with tzcode2016c (ie, our *previous* set of releases)=
:

    if (warnings && (ferror(stderr) || fclose(stderr) !=3D 0))
        return EXIT_FAILURE;

I cannot reproduce any failure locally by forcing "warnings" on, but I
speculate that for some reason fclose(stderr) fails on your platform.

We could probably persuade the IANA guys that fclose() is a bad idea here
and fflush() would be saner.  Could you determine whether "make install"
passes in that environment if you replace this code with

    if (warnings && (fflush(stderr) !=3D 0 || ferror(stderr)))
        return EXIT_FAILURE;

?

Independently of that, though, I think it's a bad idea to be using such
an environment as your build platform, because it means you are getting
different results (different package contents) depending on whether you
happen to build with the install target directory on that mount or not.

            regards, tom lane

Yes, zic has always attempted to create hard links and it never failed on
our build farm and we haven't changed anything.

The patch given doesn't help. I guess the problem is that in dolink(), the
fromfield value is a directory (path to postgresql/timezone) and hence the
make install fails at following check:
--
fromisdir = itsdir(fromfield);
if (fromisdir)
        {
                char const *e = strerror(fromisdir < 0 ? errno : EPERM);

                fprintf(stderr, _("%s: link from %s/%s failed: %s\n"),
                                progname, directory, fromfield, e);

                exit(EXIT_FAILURE);
        }
--

"fromfield" value should have been the file inside the timezone directory.

On Wed, Oct 26, 2016 at 10:11 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> > During building of the ongoing PG updates (9.6.1, 9.5.5 to 9.1.24), we
> > found that the build failed for Linux on 9.3 and 9.2 with the following
> > error:
> > ...
> > warning: "./data/backward", line 125: symbolic link used because hard
> link
> > failed: Operation not permitted
> > .*/zic: link from
> > /mnt/hgfs/pginstaller-repo/**server/staging/linux/share/
> postgresql/timezone/US/Eastern
> > failed: Operation not permitted*
> > make[2]: *** [install] Error 1
>
> > On investigating, we found that it was because of the few commits in the
> > new updates related to zic.c which tries to create hard links. These hard
> > links failed to create on our build farm because we were building the
> > source on the mount point (/mnt/hgfs/) where hard links cannot be
> created.
> > The Linux machine is a Virtual Machine hosted on VMware Fusion on the Mac
> > Server and the Fusion doesn't allow to create hard links inside the
> shared
> > location. It errors out with "Operation not permitted". Symbolic links
> can
> > be created though.
>
> I took a closer look at this.  As I said earlier, zic has always attempted
> to use hard links, and now that I look, the fallback to symlinks has been
> there right along as well.  So you were always getting those warnings on
> that platform.  What is new, apparently, is this bit at the end of zic.c's
> main() which came in with tzcode2016c (ie, our *previous* set of releases):
>
>         if (warnings && (ferror(stderr) || fclose(stderr) != 0))
>                 return EXIT_FAILURE;
>
> I cannot reproduce any failure locally by forcing "warnings" on, but I
> speculate that for some reason fclose(stderr) fails on your platform.
>
> We could probably persuade the IANA guys that fclose() is a bad idea here
> and fflush() would be saner.  Could you determine whether "make install"
> passes in that environment if you replace this code with
>
>         if (warnings && (fflush(stderr) != 0 || ferror(stderr)))
>                 return EXIT_FAILURE;
>
> ?
>
> Independently of that, though, I think it's a bad idea to be using such
> an environment as your build platform, because it means you are getting
> different results (different package contents) depending on whether you
> happen to build with the install target directory on that mount or not.
>
>                         regards, tom lane
>



--
Sandeep Thakkar

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> Yes, zic has always attempted to create hard links and it never failed on
> our build farm and we haven't changed anything.

> The patch given doesn't help. I guess the problem is that in dolink(), the
> fromfield value is a directory (path to postgresql/timezone) and hence the
> make install fails at following check:

Well, that should have failed for anyone.  Maybe you mean that it's
trying to symlink from a symlink?

            regards, tom lane

On Wed, Nov 2, 2016 at 6:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> > Yes, zic has always attempted to create hard links and it never failed on
> > our build farm and we haven't changed anything.
>
> > The patch given doesn't help. I guess the problem is that in dolink(),
> the
> > fromfield value is a directory (path to postgresql/timezone) and hence
> the
> > make install fails at following check:
>
> Well, that should have failed for anyone.  Maybe you mean that it's
> trying to symlink from a symlink?
>
> No, I mean the itsdir(fromname); is returning the non zero value and I
thought it is because it thinks the fromname is a directory and not a file.
Anyway, I observed that there is change in the itsdir() function that it
now returns '2' for the symbolic links. Hence the error. Till previous
release, it used to return '0' for symbolic links.  If hard links cannot be
used and it fallsback to symbolic links, then itsdir() should return '0'
for symbolic links. Isn't it?



>                         regards, tom lane
>



--
Sandeep Thakkar

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> On Wed, Nov 2, 2016 at 6:59 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Well, that should have failed for anyone.  Maybe you mean that it's
>> trying to symlink from a symlink?

> No, I mean the itsdir(fromname); is returning the non zero value and I
> thought it is because it thinks the fromname is a directory and not a file.
> Anyway, I observed that there is change in the itsdir() function that it
> now returns '2' for the symbolic links. Hence the error.

Yeah.  I have a query in to the tz mailing list about that, but no
answer yet.

            regards, tom lane

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> Anyway, I observed that there is change in the itsdir() function that it
> now returns '2' for the symbolic links. Hence the error. Till previous
> release, it used to return '0' for symbolic links.  If hard links cannot be
> used and it fallsback to symbolic links, then itsdir() should return '0'
> for symbolic links. Isn't it?

After some back-and-forth with the IANA crew, I've pushed a patch that
I believe will fix this, but if you could try it in your environment that
would make me feel better.  Please check not only that the symlinks get
made, but that they point somewhere sane, because that aspect of it
seems to have been broken too :-(.

I pushed the fix to HEAD only for the moment, but if you prefer testing on
a back branch, the patch should apply fine there.

BTW, it would be a really good idea if you guys were running a buildfarm
member that represented this build environment accurately, because it's
clear that no one else, anywhere, is testing either PG or the upstream
tzcode releases in such an environment.

            regards, tom lane

Sandeep Thakkar <sandeep.thakkar@enterprisedb.com> writes:
> On Fri, Nov 4, 2016 at 8:01 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> After some back-and-forth with the IANA crew, I've pushed a patch that
>> I believe will fix this, but if you could try it in your environment that
>> would make me feel better.

> With the patch, the build on 9.2 is now successful. Thanks!

Great, thanks for testing.  I'm pushed this to the back branches as well.

            regards, tom lane