Re: [nfsv4] Fw: Global namespace and multiple file access protocols

cburnett@us.ibm.com

Subject: Re: [nfsv4] Fw: Global namespace and multiple file access protocols
Message-ID: <OF65F53140.4C822B09-ON87256E14.0071A27C-86256E14.0072930B@us.ibm.com>
From: Carl Burnett <cburnett@us.ibm.com>
Date: Wed, 7 Jan 2004 14:53:34 -0600

Where I think the below model should be different from AFS, is that NFSv4 
clients should consider the NFSv4 file server as their "VLDB". That is 
they should go back to NFS servers to get the location data when they 
receive NFS4ERR_MOVED as part of a lookup sequence that does not yield a 
filehandle. The NFS servers are the constituents of the namespace 
infrastructure (the NFS version of the VLDB). While the AFS/DFS model of 
the clients going somewhere else for the location attributes worked very 
well, it did add complication to the client implementation. It also 
created additional points of failure as well as greater complication for 
administration, deployment, and problem determination. Placing more of 
this in the datacenter behind servers should be an improvement. It also 
facilitates the ideas Jon H. has posted regarding server side traffic 
management.

Carl

Carl Burnett
AIX Kernel Architecture - Network File System
(512) 838-8498, TL 678-8498
(please reply to cburnett@us.ibm.com)

Ted Anderson <TedAnderson@mindspring.com>
Sent by: nfsv4-admin@ietf.org
01/07/2004 02:19 PM

        To:     nfsv4@ietf.org
        cc: 
        Subject:        Re: [nfsv4] Fw: Global namespace and multiple file 
access protocols

The term "referral" is causing some confusion on at least two levels,
one easily clarified and the other more subtle.

The first confusion arises because NFSV4ERR_MOVED can be returned in two
situations.  In one, a transition occurs (e.g. during LOOKUP) to the
root of a new filesystem[*] which resides on another server.  The other
occurs when most any operation occurs on a file handle and the (whole)
containing filesystem has been migrated (aka relocated, moved) to
another server.  I think the recent exchange between Jon and Dave has
clearly described these two cases.

The cases differ in that the second one uses a file handle obtained from
some previous operation when the filesystem was located on the server.
The cases are related by history in the sense that after a filesystem is
moved a referral is left behind where its root was formerly located in
the parent filesystem.  But what exactly is this referral thingy?

By way of background I'd like to briefly explain how AFS (DFS is very
similar) does this.  In AFS filesystems are called volumes and the
junction between two volumes is called a mountpoint.  An AFS mountpoint
is an actual file system object, implemented as a special symbolic link,
which resides in the parent volume and contains the name of the child
volume.  The AFS client gets the name of the child volume from the
server and then contacts a separate service called the VLDB (Volume
Location Database) to locate the file server hosting this volume.  The
volume name in mountpoint can be qualified by a cell name; a cell is the
AFS term for a file system administrative domain.  The cell name is also
the second component of AFS absolute pathnames.

A key feature of this architecture is the separation of the namespace
construction, via filesystem junctions (AFS mountpoints), and the data
location, handled by a separate service (VLDB).  There are very good
reasons for making this distinction: Nico mentioned some[1] and the
recent discussion about cluster load-balancing highlights another.  The
namespace used by NFSv4 should have this same architectual separation,
even though the implementation might make this much less obvious to
NFSv4 clients than it is to AFS clients.

The NFSv4 client/server protocol does not mention referrals and probably
in many cases servers won't store them in the file system directories
instead they are represented in some other form.  One oft-mentioned
approach imagines a database exported by an LDAP server.  The database
contains two tables, one conceptually listing pathname prefixes and
filesystem names and another table mapping filesystem names to server
locations.  This two level arrangement maintains the architectural
separation described above.  Creating a new filesystem makes an entry in
the second table giving its location, while attaching the filesystem to
the namespace adds an entry to the first table.  Moving a filesystem to
another server modifies only the second table.  The first table, call it
the namespace table, logically encodes the information AFS manages with
its filesystem-resident mountpoints.  The second table, the location
table, is represented in AFS by entries in the VLDB.

The conceptual nature of these two tables and their complete absence
from the NFSv4 protocol is a source of confusion surrounding what we've
been calling referrals.  In the protocol, referrals exist only as
ephemeral file handles whose fs_locations attribute can be queried but
all other operations on them, including GETFH, fail with NFS4ERR_MOVED.
A NFSv4 server that exports several filesystems has them laced together
using pseudo filesystems.  The junctions to real exported filesystems
conceptually represent entries in the namespace table, the location
table entry only notes that the filesystem is local.  When a filesystem
is moved to another server, the location table is updated with the new
host information.  From this perspective, a referral isn't created as a
placeholder for the departed filesystem (as I implied above), the
referral was logically there all along.

We can and probably will continue using the term "referral", but
hopefully this message will provide some details and names for the
underlying concepts.  These may come in handy in further discussions on
the namespace topic.

Ted Anderson

[*] I use "filesystem", one word, to refer a singly-rooted sub-tree
     whose nodes all use the same fsid.  I use "file system" to refer to
     a collection of software and protocols.  A filesystem is mounted
     while a file system is installed.  So you could say "the ext2 file
     system is used to hold the /tmp filesystem".
[1] 
http://www.ietf.org/mail-archive/working-groups/nfsv4/current/msg00589.html

_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4