Link atomicity [was Re: NFS Question]

[jim.mcginness at att.net]

Derek, you said:
> One way of creating a lock file is done using the O_EXCL
> option to open(2).  From the manpage for that system call:
> 
>               O_EXCL is broken on NFS  file  sys­
>               tems,  programs  which  rely  on  it for performing
>               locking tasks will contain a race  condition.   The
>               solution for performing atomic file locking using a
>               lockfile is to create a unique file on the same  fs
>               (e.g., incorporating hostname and pid), use link(2)
>               to make a link to the lockfile. If  link()  returns
>               0,  the lock is successful.  Otherwise, use stat(2)
>               on the unique file to check if its link  count  has
>               increased to 2, in which case the lock is also suc­
>               cessful.

I have always told people who need portability and aren't looking for
high-performance, fine-grained locks that they should use this sort of
link mechanism. It's definitely atomic and you can implement it in shell
scripts. I've never heard of the suggestion that it would be okay to
consider the link successful if the link call returned failure but the 
link count, on a subsequent stat, was 2. That sounds like bad advice.
What I've always done is put the unique node/pid combination _in_ the
file as well. You verify that the lock is (still) yours by reading the
lockfile after acquiring the lock. You also verify that the lock is still
yours before removing the lockfile (raising an alarm if you thought you
had the lock but find that someone had taken it from you).


> However, kernel developers at MCL have told me that because NFS by
> default uses asyncronous I/O, this also contains a race condition.

I don't see this.

NFS can be hosted on top of a file system using async i/o, delayed
writes, etc, but the underlying file system is a common rendezvous
point for all NFS clients and processes running on the NFS server.
If linking is atomic on the underlying file system, it must also be
atomic for the NFS clients.

There would of course be a race condition if a process other than 
the lock owner, the process that succeeded in creating the link, 
were to unlink the lockfile. The removal could arrive immediately 
after a successful link attempt.

But perhaps what they were referring to is that changes to other 
files during the interval while the lockfile is held would not
necessarily be committed to disk at the point when the lockfile
is removed.