<?xml version='1.0' encoding='utf-8'?>

<!DOCTYPE rfc SYSTEM "rfc2629-xhtml.ent">
<?xml-stylesheet type="text/xsl" href="rfc2629.xslt"?> [
  <!ENTITY nbsp    "&#160;">
  <!ENTITY zwsp   "&#8203;">
  <!ENTITY nbhy   "&#8209;">
  <!ENTITY wj     "&#8288;">
]>
<rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-nfsv4-delstid-08" number="9754" ipr="trust200902" obsoletes=""
 scripts="Common,Latin" updates="" sortRefs="true" submissionType="IETF" consensus="true" symRefs="true" tocDepth="3" tocInclude="true" version="3" xml:lang="en">

<!-- [rfced] Please review the following questions regarding this
document's title:

a) May we update "Extending" to "Extensions for"? Also, the abstract notes
that this document presents extensions for both opening and delegating files,
but the title only mentions opening of files. Should "Opening of Files" be
updated to "Opening and Delegating Files"?

Original:
  Extending the Opening of Files in NFSv4.2

Perhaps:
  Extensions for Opening and Delegating Files in NFSv4.2

b) How may we update the document's short title (which appears in the running
header of the PDF output) for consistency with the document title?

Original:
  Deleg Stateid

Perhaps:
  NFSv4.2 Extensions

Or:
  Extending NFSv4.2
-->

<front>
  <title abbrev="Deleg Stateid">
    Extending Stateid">Extending the Opening of Files in NFSv4.2
  </title> NFSv4.2</title>
  <seriesInfo name="Internet-Draft" value="draft-ietf-nfsv4-delstid-08"/> name="RFC" value="9754"/>
  <author fullname="Thomas Haynes" initials="T." surname="Haynes">
    <organization abbrev="Hammerspace">Hammerspace</organization>
    <address>
      <email>loghyr@hammerspace.com</email>
    </address>
  </author>
  <author fullname="Trond Myklebust" initials="T." surname="Myklebust">
    <organization abbrev="Hammerspace">Hammerspace</organization>
    <address>
      <email>trondmy@hammerspace.com</email>
    </address>
  </author>
  <date year="2024" month="October" day="02"/>
  <area>Transport</area>
  <workgroup>Network File System Version 4</workgroup> year="2025" month="March"/>
  <area>WIT</area>
  <workgroup>nfsv4</workgroup>
  <keyword>NFSv4</keyword>

<!-- [rfced] FYI - We updated "for both the opening and delegating of the file
to the client" as follows. Let us know any concerns.

Original:
   This document presents several extensions for both
   the opening and delegating of the file to the client.

Updated:
   This document presents several extensions for both
   opening the file and delegating it to the client.
-->

  <abstract>
    <t>
      The Network File System v4 (NFSv4) allows a client to both open a
      file and be granted a delegation of that file.  This delegation
      provides the client the right to authoritatively cache metadata
      on the file locally.  This document presents several extensions
      for both the opening and delegating of the file and delegating it to
      the client. This document extends NFSv4.2 (see RFC7863). RFC 7863).
    </t>
  </abstract>

  <note removeInRFC="true">
    <t>
      Discussion of this draft takes place
      on the NFSv4 working group mailing list (nfsv4@ietf.org),
      which is archived at
      <eref target="https://mailarchive.ietf.org/arch/browse/nfsv4/"/>.
      Working Group information can be found at
      <eref target="https://datatracker.ietf.org/wg/nfsv4/about/"/>.
    </t>
  </note>

</front>

<middle>

<section anchor="sec_intro" numbered="true" removeInRFC="false" toc="default">
  <name>Introduction</name>

<!-- [rfced] This document contains some sentences that are difficult to
follow because they contain many parentheticals. Below are two examples
(but the document contains more):

Original:
   A compound (see Section 2.3 of [RFC8881]) with a GETATTR (see
   Section 18.7 of [RFC8881]) or READDIR (see Section 18.23 of
   [RFC8881]) can report the file's attributes without bringing the file
   online.  However, either an OPEN or a LAYOUTGET (see Section 18.43 of
   [RFC8881]) might cause the file server to retrieve the archived data
   contents, bringing the file online.
   ...
   Either type of
   stateid is sufficient to enable the server to treat the file as if it
   were open, which allows READ (See Section 18.25 of [RFC8881]), WRITE
   (See Section 18.38 of [RFC8881]), LOCK (See Section 18.12 of
   [RFC8881]), and LAYOUTGET (see Section 18.50 of [RFC8881]) operations
   to proceed.

Many of these sentences include operations (e.g., GETATTR and READ), and the
parentheticals point to sections in RFC 8881 that define those operations. To
improve readability of sentences like this, we recommend adding the operations
to the Definitions section (i.e, Section 1.1) and then omitting the
parentheticals from the sentences.

You may also consider doing the same for other terminology from [RFC8881],
e.g., delegation, stateid, compound, parallel NFS (pNFS), change attribute,
time_metadata attribute, time_access attribute, time_modify attribute, and
NFS4ERR_DELAY.

Let us know your thoughts.

Perhaps (add to Section 1.1; sentence form with no section pointers):
  The following operations are used in this document as defined in [RFC8881]:
  CB_GETATTR, CLOSE, DELEGRETURN, GETATTR, LAYOUTGET, LOCK, OPEN,
  READ, READDIR, SETATTR, and WRITE.

Or (add to Section 1.1; <dl> with section pointers):
  The following operations are used in this document as defined in [RFC8881]:

  CB_GETATTR: Section 20.1 of [RFC8881]

  CLOSE: Section 18.2 of [RFC8881]

  DELEGRETURN: Section 18.6 of [RFC8881]

  GETATTR: Section 18.7 of [RFC8881]

  LAYOUTGET: Section 18.43 of [RFC8881]

  LOCK: Section 18.10 of [RFC8881]

  OPEN: Section 18.16.1 of [RFC8881]

  READ: Section 18.22 of [RFC8881]

  READDIR: Section 18.23 of [RFC8881]

  SETATTR: Section 18.30 of [RFC8881]

  WRITE: Section 18.32 of [RFC8881]
-->

  <t>
    In the Network File System version4 version 4 (NFSv4), a client may be granted a
    delegation for a file (see Section 1.8.4 of <xref target="RFC8881"
    format="default" section="1.8.4"
    sectionFormat="of"/>). This allows the client to act as the authority for
    the file's metadata data and data. metadata. This document presents a number of
    extensions which that enhance the functionality of opens and delegations. These
    allow the client to:
  </t>

<!-- [rfced] We moved the parenthetical to appear after "delegation stateid"
in this sentence. Section 8.2 of RFC 8881 defines stateids and mentions
both "delegation stateid" and "OPEN stateid". Let us know any concerns.

Original:
   *  during the OPEN procedure, retrieve either the open stateid (see
      Section 8.2 of [RFC8881]) or delegation stateid, but not both
      simultaneously.

Updated:
   *  retrieve either the open or delegation stateid (see
      Section 8.2 of [RFC8881]), but not both simultaneously, during the OPEN
      procedure; and
-->

  <ul spacing="normal">
    <li>
      detect
    <li>detect an offline file, which may require significant effort to obtain.
    </li>
    <li>
      determine obtain;</li>

    <li>determine which extensions of OPEN flags (see Section 18.16 of <xref target="RFC8881" format="default"
    section="18.16" sectionFormat="of"/>) flags are supported by the server.
    </li>
    <li>
      during the OPEN procedure, retrieve server;</li>

    <li>retrieve either the open or delegation stateid (see Section 8.2 of <xref target="RFC8881"
      format="default" sectionFormat="of"/>) or
      delegation stateid,
    section="8.2" sectionFormat="of"/>), but not
    both simultaneously.
    </li>
    <li>
      cache simultaneously, during the OPEN procedure; and</li>

    <li>cache both the access and modify timestamps, thereby reducing the
    frequency with which the client must query the server for this
    information.
    </li>
  </ul>

  <t>
    Using the process detailed in <xref target="RFC8178" format="default"
    sectionFormat="of"/>, the revisions in this document become an extension
    of NFSv4.2 <xref target="RFC7862" format="default"
    sectionFormat="of"/>. They are built on top of the external data
    representation External Data
    Representation (XDR) <xref target="RFC4506" format="default"
    sectionFormat="of"/> generated from <xref target="RFC7863"
    format="default" sectionFormat="of"/>.
  </t>

  <section anchor="sec_defs" numbered="true" removeInRFC="false"  toc="default">
    <name>Definitions</name>
    <t>This document uses the following terminology:</t>
    <dl newline="false" spacing="normal">
      <dt>offline file:</dt>
      <dd>
        A file which that exists on a device which that is not connected to the
        server. There is typically a cost associated with bringing the
        file to an online status. Historically Historically, this would be a file on
        tape media media, and the cost would have been finding and loading the
        tape. A more modern interpretation is that the file is in the
        cloud
        cloud, and the cost is a monetary one in downloading the file.
      </dd>
      <dt>proxy:</dt>
      <dd>
        Proxying
        The proxying of attributes occurs when a client has the authority, as
        granted by the appropriate delegation, to represent the attributes
        normally maintained by the server.  For read attributes, this
        occurs when the client has either a read or write delegations delegation
        for the file. For write attributes, this occurs when the client
        has a write delegation for the file.  The client having this
        authority is the "proxy" for those attributes.
      </dd>
    </dl>
  </section>
  <section numbered="true" removeInRFC="false"  toc="default">
    <name>Requirements Language</name>
        <t>
    The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>",
    "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL NOT</bcp14>",
    "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>",
    "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
    "<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document are to be
    interpreted as described in BCP&nbsp;14 <xref
      target="RFC2119" format="default" sectionFormat="of"/> target="RFC2119"/> <xref
      target="RFC8174" format="default" sectionFormat="of"/>
    target="RFC8174"/> when, and only when, they appear in all capitals, as
    shown here.
        </t>
  </section>
</section>

<section anchor="sec_offline" numbered="true" removeInRFC="false"  toc="default">
  <name>Offline Files</name>
  <t>
    If a file is offline, then the server has immediate high-performance
    access to the file's attributes, but not to the file's content.
    The action of retrieving the data content is expensive, to the extent
    that the content should only be retrieved if it is going to be used.
    For example, a graphical file manager (such as OSX's Finder) Finder in Mac OS X) may
    want to access the beginning of the file to preview it for an a user
    who is hovering their pointer over the file name and not accessing
    it otherwise.  If the file is retrieved, it will most likely either be either
    immediately thrown away or returned.
  </t>

<!-- [rfced] We updated the section numbers below for accuracy with RFC 8881.
Please review and confirm that these changes are correct.

Original:
   Either type of stateid is sufficient to enable the server to treat the file
   as if it were open, which allows READ (See Section 18.25 of [RFC8881]),
   WRITE (See Section 18.38 of [RFC8881]), LOCK (See Section 18.12 of
   [RFC8881]), and LAYOUTGET (see Section 18.50 of [RFC8881]) operations to
   proceed.

Updated:
   Either type of stateid is sufficient to enable the server to treat the file
   as if it were open, which allows READ (see Section 18.22 of [RFC8881]), WRITE
   (see Section 18.32 of [RFC8881]), LOCK (see Section 18.10 of
   [RFC8881]), and LAYOUTGET (see Section 18.43 of [RFC8881]) operations
   to proceed.
-->

<!-- [rfced] Is a word missing in "filehandle to the data content"?

Original:
   For non-parallel NFS (pNFS)
   systems (see Section 12 of [RFC8881]) , the OPEN operation requires a
   filehandle to the data content.

Perhaps:
   For non-parallel NFS
   systems (see Section 12 of [RFC8881]), the OPEN operation requires a
   filehandle to retrieve the data content.
-->

  <t>
    A compound (see Section 2.3 of <xref target="RFC8881" format="default" section="2.3" sectionFormat="of"/>)
    with a GETATTR (see Section 18.7 of <xref target="RFC8881" format="default" section="18.7"
    sectionFormat="of"/>) or READDIR (see Section 18.23 of <xref target="RFC8881" format="default"
    section="18.23" sectionFormat="of"/>) can report the file's attributes
    without bringing the file online.  However, either an OPEN or a LAYOUTGET
    (see Section 18.43 of <xref target="RFC8881" format="default" section="18.43" sectionFormat="of"/>) might
    cause the file server to retrieve the archived data contents, bringing the
    file online.  For non-parallel NFS (pNFS) systems (see Section 12 of <xref
    target="RFC8881"
    format="default" sectionFormat="of"/>) , section="12" sectionFormat="of"/>), the OPEN operation
    requires a filehandle to the data content. For pNFS parallel NFS (pNFS) systems, the
    filehandle retrieved from an OPEN need not cause the data content to be
    retrieved. But However, when the LAYOUTGET operation is processed, a layout type specific layout-type-specific mapping
     will cause the data content to be retrieved from offline
    storage.
  </t>
  <t>
    If the client is not aware that the file is offline, it might
    inadvertently open the file to determine what type of file it
    is accessing.  By interrogating the new attribute fattr4_offline,
    a client can predetermine the availability of the file, avoiding the
    need to open it at all. Being offline might also involve situations
    in which the file is archived in the cloud, i.e., there can be an
    expense in both retrieving the file to bring it online and in sending
    the file back to offline status.
  </t>

  <section anchor="ssec_offline_attr" numbered="true" removeInRFC="false"  toc="default">
    <name>XDR for the Offline Attribute</name>

<sourcecode name="" type="" type="xdr" markers="true"><![CDATA[
///
/// typedef bool            fattr4_offline;
///
///
/// const FATTR4_OFFLINE            = 83;
///
]]>
    </sourcecode>
]]></sourcecode>

  </section>
</section>

<section anchor="ssec_open_xor_xdr" numbered="true" removeInRFC="false"  toc="default">
  <name>Determining OPEN Feature Support</name>

<!-- [rfced] May we update "[RFC8178] (see Section 4.4.2)" as follows? Or do
you prefer the original?

Original:
   [RFC8178] (see Section 4.4.2) allows for extending a particular minor
   version of the NFSv4 protocol without requiring the definition of a
   new minor version.

Perhaps:
   Section 4.4.2 of [RFC8178] allows for extending a particular minor
   version of the NFSv4 protocol without requiring the definition of a
   new minor version.
-->

  <t>
    <xref target="RFC8178" format="default" sectionFormat="of"/> (see Section 4.4.2) <xref
    target="RFC8178" section="4.4.2" sectionFormat="bare"/>) allows for
    extending a particular minor version of the NFSv4 protocol without
    requiring the definition of a new minor version.  The client can
    probe the capabilities of the server and and, based on the result, determine if
    both it and the server support optional features not previously specified
    as part of the minor version.
  </t>

  <t>
    The fattr4_open_arguments attribute is a new XDR extension which that
    provides helpful support when the OPEN procedure is extended in
    such a fashion. It models all of the parameters via bitmap4 data
    structures, which allows for the addition of a new flag to any of
    the OPEN arguments (see Section 18.16.1 of <xref target="RFC8881"
    format="default" section="18.16.1" sectionFormat="of"/>). The scope of this attribute
    applies to all objects with a matching fsid.
  </t>
  <t>
    Two new flags are provided:
  </t>
  <ul spacing="normal">
    <li>
      OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION
      (see <xref target="sec_open_xor" format="default" sectionFormat="of"/>)
    </li>
    <li>
      OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS
      (see <xref target="sec_proxy" format="default" sectionFormat="of"/>)
    </li>
  </ul>
  <t>
    Subsequent extensions can use this framework when introducing new
    <bcp14>OPTIONAL</bcp14> functionality to OPEN, OPEN by creating a new
    flag for each <bcp14>OPTIONAL</bcp14> parameter.
  </t>
  <t>
    Since fattr4_open_arguments is a <bcp14>RECOMMENDED</bcp14> attribute, if the
    server informs the client via NFS4ERR_ATTRNOTSUPP that it does not support this new
    attribute, the client <bcp14>MUST</bcp14> take this to mean that
    the additional new <bcp14>OPTIONAL</bcp14> functionality to OPEN
    is also not supported.
  </t>
  <t>
    Some other concerns are how to process both currently
    <bcp14>REQUIRED</bcp14> flags and <bcp14>OPTIONAL</bcp14> flags
    which that
    become <bcp14>REQUIRED</bcp14> in the future.  The server
    <bcp14>MUST</bcp14> mark <bcp14>REQUIRED</bcp14> flags as being supported.
    Note that these flags <bcp14>MUST</bcp14> only change from
    <bcp14>OPTIONAL</bcp14> to <bcp14>REQUIRED</bcp14> when the NFSv4 minor
    version is incremented.
  </t>

  <section anchor="ssec_open_xdr" numbered="true" removeInRFC="false"  toc="default">
    <name>XDR for Open Arguments</name>

<sourcecode name="" type="" type="xdr" markers="true"><![CDATA[
///
/// struct open_arguments4 {
///   bitmap4  oa_share_access;
///   bitmap4  oa_share_deny;
///   bitmap4  oa_share_access_want;
///   bitmap4  oa_open_claim;
///   bitmap4  oa_create_mode;
/// };
///
///
/// enum open_args_share_access4 {
///    OPEN_ARGS_SHARE_ACCESS_READ  = 1,
///    OPEN_ARGS_SHARE_ACCESS_WRITE = 2,
///    OPEN_ARGS_SHARE_ACCESS_BOTH  = 3
/// };
///
///
/// enum open_args_share_deny4 {
///    OPEN_ARGS_SHARE_DENY_NONE  = 0,
///    OPEN_ARGS_SHARE_DENY_READ  = 1,
///    OPEN_ARGS_SHARE_DENY_WRITE = 2,
///    OPEN_ARGS_SHARE_DENY_BOTH  = 3
/// };
///
///
/// enum open_args_share_access_want4 {
///    OPEN_ARGS_SHARE_ACCESS_WANT_ANY_DELEG           = 3,
///    OPEN_ARGS_SHARE_ACCESS_WANT_NO_DELEG            = 4,
///    OPEN_ARGS_SHARE_ACCESS_WANT_CANCEL              = 5,
///    OPEN_ARGS_SHARE_ACCESS_WANT_SIGNAL_DELEG_WHEN_RESRC_AVAIL
///                                                    = 17,
///    OPEN_ARGS_SHARE_ACCESS_WANT_PUSH_DELEG_WHEN_UNCONTENDED
///                                                    = 18,
///    OPEN_ARGS_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS    = 20,
///    OPEN_ARGS_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION = 21
/// };
///
///
/// enum open_args_open_claim4 {
///    OPEN_ARGS_OPEN_CLAIM_NULL          = 0,
///    OPEN_ARGS_OPEN_CLAIM_PREVIOUS      = 1,
///    OPEN_ARGS_OPEN_CLAIM_DELEGATE_CUR  = 2,
///    OPEN_ARGS_OPEN_CLAIM_DELEGATE_PREV = 3,
///    OPEN_ARGS_OPEN_CLAIM_FH            = 4,
///    OPEN_ARGS_OPEN_CLAIM_DELEG_CUR_FH  = 5,
///    OPEN_ARGS_OPEN_CLAIM_DELEG_PREV_FH = 6
/// };
///
///
/// enum open_args_createmode4 {
///    OPEN_ARGS_CREATEMODE_UNCHECKED4     = 0,
///    OPEN_ARGS_CREATE_MODE_GUARDED       = 1,
///    OPEN_ARGS_CREATEMODE_EXCLUSIVE4     = 2,
///    OPEN_ARGS_CREATE_MODE_EXCLUSIVE4_1  = 3
/// };
///
///
/// typedef open_arguments4 fattr4_open_arguments;
///
///
/// %/*
/// % * Determine what OPEN supports.
/// % */
/// const FATTR4_OPEN_ARGUMENTS     = 86;
///
///
/// const OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION = 0x200000;
///
///
/// const OPEN4_RESULT_NO_OPEN_STATEID = 0x00000010;
///
 ]]>
    </sourcecode>
]]></sourcecode>

  </section>
</section>

<section anchor="sec_open_xor" numbered="true" removeInRFC="false"  toc="default">
  <name>OPEN Grants Only One of Open or Delegation Stateid</name>

<!-- [rfced] May we update "only one of" in the title of Section 4 as follows?

Original:
  4.  OPEN grants only one of Open or Delegation Stateid</name> Stateid

Perhaps:
  4.  OPEN Grants Either an Open or a Delegation Stateid
-->

<t>
    The OPEN (See Section 18.16 of (see <xref target="RFC8881" format="default" section="18.16" sectionFormat="of"/>)
    procedure returns an open stateid to the client to reference the state of
    the file.  The client could also request a delegation stateid in the OPEN
    arguments.  The file can be considered open for the client as long as the
    count of open and delegated stateids is greater than 0.  Either type of
    stateid is sufficient to enable the server to treat the file as if it were
    open, which allows READ (See Section 18.25 of (see <xref target="RFC8881"
    format="default" section="18.22"
    sectionFormat="of"/>), WRITE (See Section 18.38
    of (see <xref target="RFC8881" format="default" section="18.32"
    sectionFormat="of"/>), LOCK (See Section 18.12 of (see <xref target="RFC8881" format="default" section="18.10"
    sectionFormat="of"/>), and LAYOUTGET (see Section 18.50 of <xref target="RFC8881" format="default"
    section="18.43" sectionFormat="of"/>) operations to proceed. If the client
    gets both an open and a delegation stateid as part of the OPEN, then it
    has to return them both to the server.  A further consideration is that
    during each operation, the client can send a costly GETATTR (See Section 18.7 of (see <xref
    target="RFC8881"
    format="default" section="18.7" sectionFormat="of"/>).
  </t>

  <t>
    If the client knows that the server supports the
    OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION flag (as determined by an
    earlier GETATTR operation which that queried for the fattr4_open_arguments
    attribute), then the client can supply that flag during the OPEN and only
    get either an open or a delegation stateid.
  </t>

<!-- [rfced] We do not see "OPEN4resok.stateid" in Section 18.16.2 of RFC 8811.
Should this be updated to "OPEN4resok"?

Original:
   The open stateid field,
   OPEN4resok.stateid (see Section 18.16.2 of [RFC8881]), MUST be set to
   the special all zero stateid in this case.

Perhaps:
   The open stateid field,
   OPEN4resok (see Section 18.16.2 of [RFC8881]), MUST be set to
   the special all-zero stateid in this case.
-->

  <t>
    The client is already prepared to not get a delegation
    stateid
    stateid, even if requested. In order to not send an open
    stateid, the server <bcp14>MUST</bcp14> indicate that fact with the result
    flag of OPEN4_RESULT_NO_OPEN_STATEID. The open stateid field,
    OPEN4resok.stateid (see Section 18.16.2 of <xref target="RFC8881"
    format="default" section="18.16.2" sectionFormat="of"/>), <bcp14>MUST</bcp14> be set to the
    special all zero all-zero stateid in this case.
  </t>

<!-- [rfced] Would it be helpful to update "then the server has three options"
in one of the following ways?

Original:
   If the client then opens the file for read-write (with
   OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION set), then the server has
   three options:

   1.  Only an open stateid with the correct seqid.

   2.  Only a delegation stateid with the open stateid now having an
       incorrect seqid as it needs to be upgraded.

   3.  Both an open (which will be upgraded) and a delegation stateid.

Perhaps:
   If the client then opens the file for read-write (with
   OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION set), the server
   can return one of the following three options:

   1.  Only an open stateid with the correct seqid.

   2.  Only a delegation stateid with the open stateid now having an
       incorrect seqid as it needs to be upgraded.

   3.  Both an open stateid (which will be upgraded) and a delegation stateid.

Or:
   If the client then opens the file for read-write (with
   OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION set), then the server has
   three options. The server can return:

   1.  only an open stateid with the correct seqid,

   2.  only a delegation stateid (with the open stateid now having an
       incorrect seqid as it needs to be upgraded), or

   3.  both an open stateid (which will be upgraded) and a delegation
       stateid.
-->

<t>
    Note that the OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION flag is a
    hint. The server might return both stateids. Consider the scenario in
    which the client opens a file for read-only (with
    OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION set) and gets only gets an open
    stateid. If the client then opens the file for read-write (with
    OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION set), then the server has
    three options:
  </t>
  <ol>
    <li>Only an open stateid with the correct seqid.</li>
    <li>Only a delegation stateid with the open stateid now having an incorrect seqid as it needs to be upgraded.</li>
    <li>Both an open stateid (which will be upgraded) and a delegation stateid.</li>
  </ol>
  <t>
    In this scenario, returning just a delegation stateid would hide
    information from the client. If the client already has an open stateid,
    then the server <bcp14>SHOULD</bcp14> ignore the
    OPEN4_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION flag and return both the open
    and delegation stateids.
  </t>

  <section anchor="delstid_analysis" numbered="true" removeInRFC="false"  toc="default">
  <name>Implementation Experience</name>
    <t>
      The CLOSE operation (see Section
      18.2 of <xref target="RFC8881" format="default" section="18.2" sectionFormat="of"/>)
      neither explicitly nor implicitly releases any
      delegation stateids. This is not symmetrical with the OPEN operation,
      which can grant both an open and a delegation stateid.  This specification
      could have tried to extend the CLOSE operation to release both
      stateids, but implementation experience shows that is more costly
      than the approach which that has been proposed.
    </t>
    <t>

<!-- [rfced] Please review "That takes" in the second sentence below. Would
updating to "This involves" be an improvement? Also, would it be helpful
to update the long third sentence to be a bulleted list?

Original:
   Consider a small workload of creating a file with content.  That
   takes 3 synchronous and 1 asynchronous operations with existing
   implementations. The first synchronous one has to OPEN the file, the
   second synchronous one performs the WRITE to the file, the third
   synchronous one has to CLOSE the file, and the fourth asynchronous
   one uses DELEGRETURN (see Section 18.6 of [RFC8881]) to return the
   delegation stateid.

Perhaps:
   Consider a small workload of creating a file with content. This
   involves three synchronous operations and one asynchronous operation with existing
   implementations:

   * The first synchronous operation has to OPEN the file.

   * The second synchronous operation performs the WRITE to the file.

   * The third synchronous operation has to CLOSE the file.

   * The asynchronous operation uses DELEGRETURN (see Section 18.6 of [RFC8881]) to return
     the delegation stateid.
-->

    <t>
      Consider a small workload of creating a file with content. That takes three
      synchronous operations and one asynchronous operation with existing
      implementations.
       The first synchronous operation has to OPEN the file;
      the second synchronous one performs the WRITE to the file;
      the third synchronous one has to CLOSE the file; and
      the asynchronous one uses DELEGRETURN (see <xref target="RFC8881" format="default"
      section="18.6" sectionFormat="of"/>) to return the delegation stateid.
    </t>

    <sourcecode name="" type="" markers="true"><![CDATA[
      SEQ PUTFH OPEN GETFH GETATTR
      SEQ PUTFH WRITE GETATTR
      SEQ PUTFH CLOSE
      ...
      SEQ PUTFH DELEGRETURN
]]>
    </sourcecode>
]]></sourcecode>
    <t>
      With the proposed approach of setting the
      OPEN_ARGS_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION flag during
      the OPEN, the number of operations is always 3. three.  The first two
      compounds are still synchronous, but the last is asynchronous. I.e., That is,
      since the client no longer has to send a CLOSE operation, it can
      delay the DELEGRETURN until either the server requests it back
      via delegation recall or garbage collection causes the client to
      return the stateid.
    </t>
    <sourcecode name="" type="" markers="true"><![CDATA[
      SEQ PUTFH OPEN(OPEN_ARGS_SHARE_ACCESS_WANT_OPEN_XOR_DELEGATION)
          GETFH GETATTR
      SEQ PUTFH WRITE GETATTR
      ...
      SEQ PUTFH DELEGRETURN
]]>
    </sourcecode>
]]></sourcecode>
    <t>
      This approach reduces the cost of synchronous operations by 33%
      and the total number of operations by 25%. Contrast that against with
      the alternative proposal of having CLOSE return both stateids,
      which would not reduce the number of synchronous operations.
    </t>
  </section>
</section>

<section anchor="sec_proxy" numbered="true" removeInRFC="false"  toc="default">
  <name>Proxying of Times</name>

<!-- [rfced] For readability, may we adjust the text below as follows? In the
suggested text below, we moved the phrase "to notify...values" to the
beginning of the sentence and used a semicolon to split up the long
sentence.

Original:
   While the client could send a compound of the form: SEQ, PUTFH,
   SETATTR (time_modify | time_access), DELEGRETURN, to notify the
   server of the proxied values, that SETATTR (see Section 18.30 of
   [RFC8881]) operation would cause either or both of change (see
   Section 5.8.1.4 of [RFC8881]) or time_metadata (see Section 5.8.2.42
   of [RFC8881]) to be modified to the current time on the server.

Perhaps:
   To notify the server of the proxied values, the client could send a
   compound of the form SEQ, PUTFH, SETATTR (time_modify |
   time_access), DELEGRETURN; however, the SETATTR operation (see
   Section 18.30 of [RFC8881]) would cause either or both of the change
   attribute (see Section 5.8.1.4 of [RFC8881]) or time_metadata
   attribute (see Section 5.8.2.42 of [RFC8881]) to be modified to the
   current time on the server.
-->

<!-- [rfced] Is "pass these times up" correct, or should this be updated to
"pass on these times"?

Original:
   As a result, it can not pass
   these times up to an application expecting POSIX compliance, as is
   often necessary for correct operation.

Perhaps:
   As a result, it cannot pass on
   these times to an application expecting POSIX compliance, as is
   often necessary for correct operation.
-->

<t>
    When a client is granted a write delegation on a file, it becomes the
    authority for the file contents and associated attributes.  If the server
    queries the client as to the state of the file via a CB_GETATTR (see Section 20.1 of <xref
    target="RFC8881" format="default" section="20.1" sectionFormat="of"/>), then, then according to
    the unextended NFSv4 protocol, it can only determine the size of the file
    and the change attribute. In the case of the client holding the
    delegation, it has the current values of the access and modify times.
    There is no way that other clients can have access to these values.  While  To
    notify the server of the proxied values, the client could send a compound
    of the form: form SEQ, PUTFH, SETATTR (time_modify | time_access), DELEGRETURN, to notify DELEGRETURN;
    however, the server of the proxied values, that SETATTR operation (see Section 18.30 of <xref target="RFC8881" format="default" section="18.30"
    sectionFormat="of"/>) operation would cause either or both of the change attribute
    (see Section 5.8.1.4 of <xref target="RFC8881" format="default"
    sectionFormat="of"/>) sectionFormat="of" section="5.8.1.4"/>) or
    time_metadata attribute (see Section 5.8.2.42 of <xref target="RFC8881" format="default" section="5.8.2.42"
    sectionFormat="of"/>) to be modified to the current time on the server.

    There is no current provision to obtain these values before
    delegation return using CB_GETATTR. As a result, it can not cannot pass these
    times up to an application expecting POSIX compliance, as is often
    necessary for correct operation.
  </t>
  <t>
    With the addition of the new flag:
    OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS,
    OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS flag, the client and server can
    negotiate that the client will be the authority for these values values, and upon
    return of the delegation stateid via a DELEGRETURN (see section
    18.6 of <xref
    target="RFC8881" format="default" section="18.6" sectionFormat="of"/>), the times will be
    passed back to the server. If the server is queried by another client for
    either the size or the times, it will need to use a CB_GETATTR to query
    the client which that holds the delegation (see Section 20.1 of <xref target="RFC8881" format="default"
    section="20.1" sectionFormat="of"/>).
  </t>

<!-- [rfced] May we update this sentence as follows to improve clarity? Note
that the suggested text a) clarifies what "it" refers to, b) revises
"fattr4_time_deleg_access attribute and fattr4_time_deleg_modify
attribute changes", and c) updates "or reject" to ", or it MUST reject".

Original:
   Further, when it gets a SETATTR with those attributes being set, then
   it MUST accept those fattr4_time_deleg_access attribute and
   fattr4_time_deleg_modify attribute changes and derive the change time
   or reject the changes with NFS4ERR_DELAY (see Section 15.1.1.3 of
   [RFC8881]).

Perhaps:
   Further, when a server gets a SETATTR with those attributes set, then
   it MUST accept those changes in the fattr4_time_deleg_access and
   fattr4_time_deleg_modify attributes and derive the change time,
   or it MUST reject the changes with NFS4ERR_DELAY (see Section 15.1.1.3 of
   [RFC8881]).
-->

  <t>
    If a server informs the client via the fattr4_open_arguments attribute
    that it supports OPEN_ARGS_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS and it
    returns a valid delegation stateid for an OPEN operation which that sets the
    OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS flag, then it <bcp14>MUST</bcp14>
    query the client via a CB_GETATTR for the fattr4_time_deleg_access
    attribute (see <xref target="ssec_proxy_xdr"/>)
    attribute and the
    fattr4_time_deleg_modify attribute (see <xref
    target="ssec_proxy_xdr"/>). (The (Note that the change time can be derived from
    the modify time.) Further, when it gets a SETATTR with those attributes being
    set, then it <bcp14>MUST</bcp14> accept those
    fattr4_time_deleg_access attribute and fattr4_time_deleg_modify
    attribute changes and derive the change time or reject the changes with
    NFS4ERR_DELAY (see Section 15.1.1.3 of <xref target="RFC8881"
    format="default" section="15.1.1.3"
    sectionFormat="of"/>).
  </t>

<!-- [rfced] We are having trouble parsing the text after "either". Please
clarify.

Original:
   The SETATTR SHOULD either be in a
   separate compound before the one containing the DELEGRETURN or when
   in the same compound, as an operation before the DELEGRETURN.

Perhaps:
   The SETATTR SHOULD be either 1) in a
   separate compound before the one containing the DELEGRETURN or 2)
   in the same compound as an operation before the DELEGRETURN.
-->

  <t>
    These new attributes are invalid to be used with GETATTR, VERIFY, and NVERIFY NVERIFY, and they
    can only be used with CB_GETATTR and SETATTR by a  client holding an
    appropriate delegation.  The SETATTR <bcp14>SHOULD</bcp14> either be
    in a separate compound before the one containing the DELEGRETURN or
    when in the same compound, as an operation before the DELEGRETURN.
    Failure to properly sequence the operations may lead to race conditions.
  </t>
  <t>
    A key prerequisite of this approach is that the server and client are in
    time synchronization with each other.  Note that while the base NFSv4.2
    does not require such synchronization, the use of RPCSEC_GSS typically
    makes such a requirement. When the client presents either the
    fattr4_time_deleg_access or the fattr4_time_deleg_modify attributes attribute to the
    server, the server <bcp14>MUST</bcp14> decide for both of them whether the time presented is:</t>

<!-- [rfced] FYI - We have reworked the text below to be a bulleted list for
ease of the reader. Please review.

Original:
   When the client presents either
   fattr4_time_deleg_access or fattr4_time_deleg_modify attributes to
   the server, the server MUST decide for both of them whether the time
   presented is before the corresponding time_access (see
   Section 5.8.2.37 of <xref target="RFC8881" format="default"
    sectionFormat="of"/>) [RFC8881]) or time_modify (see Section 5.8.2.43
   of <xref target="RFC8881" format="default"
    sectionFormat="of"/>) [RFC8881]) attribute on the file or past the current server time.

Current:
   When the client presents either
   the fattr4_time_deleg_access or the fattr4_time_deleg_modify
   attributes to the server, the server MUST decide for both of them
   whether the time presented is:

   *  before the corresponding time_access attribute (see Section 5.8.2.37 of
      [RFC8881]) or time_modify attribute (see Section 5.8.2.43 of [RFC8881])
      on the file, or

   *  past the current server time.
-->

<ul>
      <li>before the corresponding time_access attribute (see <xref target="RFC8881" section="5.8.2.37" sectionFormat="of"/>) or time_modify attribute
      (see <xref target="RFC8881" section="5.8.2.43" sectionFormat="of"/>) on the file, or</li>
      <li>past the current server time.</li>
    </ul>

    <t>When the time
    presented is before the original time, then the update is ignored. When
    the time presented is in the future, the server can either clamp the new
    time to the current time, time or it may return NFS4ERR_DELAY to the client,
    allowing it to retry.  Note that if the clock skew is large, the delay
    approach would result in access to the file being denied until the clock
    skew is exceeded.
  </t>
  <t>
    A change in the access time <bcp14>MUST NOT</bcp14> advance the change
    time, also known as the time_metadata attribute (see Section 5.8.2.42 of <xref
    target="RFC8881" format="default" sectionFormat="of"/>), but section="5.8.2.42" sectionFormat="of"/>). However, a
    change in the modify time might advance the change time (and in turn turn, the
    change attribute (See Section 5.8.1.4 of attribute; see <xref target="RFC8881"
    format="default" section="5.8.1.4"
    sectionFormat="of"/>).  If the modify time is greater than the change time
    and before the current time, then the change time is adjusted to the
    modify time and not the current time (as is most likely done on most
    SETATTR calls that change the metadata). If the modify time is in the
    future, it will be clamped to the current time.
  </t>

  <t>
    Note that each of the possible times, access, times (access, modify, and change, change) are
    compared to the current time. They should all be compared against
    the same time value for the current time. I.e., time (i.e., they do not retrieve
    a different value of the current time for each calculation. calculation).
  </t>
  <t>
    If the client sets the OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS
    flag in an OPEN operation, then it <bcp14>MUST</bcp14> support
    the fattr4_time_deleg_access
    and fattr4_time_deleg_modify attributes both in both the CB_GETATTR
    and SETATTR operations.
  </t>

  <section anchor="ssec_proxy_use" numbered="true" removeInRFC="false"  toc="default">
    <name>Use case: Case for NFSv3 client proxy</name> Client Proxy</name>
    <t>
      Consider a an NFSv3 client which that wants to access data on a server
      which
      that only supports NFSv4.2.  An implementation may introduce an
      NFSv3 server that functions as an NFSv4.2 client, serving as a
      gateway between the two otherwise incompatible systems.  As NFSv3
      is a stateless protocol, the state is not kept on the client, but rather
      on the NFSv3 server. As the NFSv3 server is already managing the
      state, it can proxy file delegations to avoid spurious GETATTRs.
      I.e., That is, as the client queries the NFSv3 server for the attributes,
      they can be served without the NFSv3 server sending a GETATTR to
      the NFSv4.2 server.
    </t>
  </section>

  <section anchor="ssec_proxy_xdr" numbered="true" removeInRFC="false"  toc="default">
    <name>XDR for Proxying of Times</name>
    <sourcecode name="" type="" type="xdr" markers="true"><![CDATA[
///
/// /*
///  * attributes for the delegation times being
///  * cached and served by the "client"
///  */
/// typedef nfstime4        fattr4_time_deleg_access;
/// typedef nfstime4        fattr4_time_deleg_modify;
///
///
/// %/*
/// % * New RECOMMENDED Attribute for
/// % * delegation caching of times
/// % */
/// const FATTR4_TIME_DELEG_ACCESS  = 84;
/// const FATTR4_TIME_DELEG_MODIFY  = 85;
///
///
/// const OPEN4_SHARE_ACCESS_WANT_DELEG_TIMESTAMPS = 0x100000;
///
 ]]>
    </sourcecode>
]]></sourcecode>
  </section>
</section>

<section anchor="xdr_desc" numbered="true" removeInRFC="false"  toc="default">
  <name>Extraction of XDR</name>
  <t>
    This document contains the external data representation (XDR) XDR <xref
    target="RFC4506" format="default" sectionFormat="of"/> description of the
    new open flags for delegating the file to the client.  The XDR description
    is embedded in this document in a way that makes it simple for the reader
    to extract into a ready-to-compile form.  The reader can feed this
    document into the following shell script to produce the machine readable machine-readable
    XDR description of the new flags:
  </t>

  <sourcecode name="" type="" markers="true"><![CDATA[
#!/bin/sh
grep '^ *///' $* | sed 's?^ */// ??' | sed 's?^ *///$??'
    ]]>
  </sourcecode>
]]></sourcecode>

  <t>
    That is, if the above script is stored in a file called "extract.sh", "extract.sh" and
    this document is in a file called "spec.txt", then the reader can do: do the
    following:
  </t>

  <sourcecode name="" type="" markers="true"><![CDATA[
sh extract.sh < spec.txt > delstid_prot.x
    ]]>
  </sourcecode>
]]></sourcecode>

  <t>
    The effect of the script is to remove leading white space from each
    line, plus a sentinel sequence of "///".  XDR descriptions with the
    sentinel sequence are embedded throughout the document.
  </t>

  <t>
    Note that the XDR code contained in this document depends on types
    from the NFSv4.2 nfs4_prot.x file (generated from <xref target="RFC7863"
    format="default" sectionFormat="of"/>).  This includes both nfs types that
    end with a 4, such 4 (such as offset4,
    length4, etc., offset4 and length4) as well as more generic
    types such (such as uint32_t and
    uint64_t. uint64_t).
  </t>
  <t>
    While the XDR can be appended to that from
    <xref target="RFC7863" format="default" sectionFormat="of"/>,
    the various code snippets belong in their respective areas of
    that XDR.
  </t>
</section>

<section anchor="sec_security" numbered="true" removeInRFC="false" toc="default">
  <name>Security Considerations</name>
  <t>
    While we are extending this document extends some capabilities for client delegation, there are
    no new security concerns.  The client cannot be queried by other clients
    as to the cached attributes. The client could report false data for the
    cached attributes, but it already has this ability via a SETATTR operation
    (see Section
    18.30 of <xref target="RFC8881" format="default" section="18.30" sectionFormat="of"/>).
  </t>
</section>

<section anchor="sec_iana" numbered="true" removeInRFC="false" toc="default">
  <name>IANA Considerations</name>
  <t>
    There are
  <t>This document has no IANA considerations.
  </t> actions.</t>
</section>

</middle>
<back>

<references>
  <name>References</name>

  <references>
  <name>Normative References</name>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.4506.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4506.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.7862.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7862.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.7863.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7863.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.8178.xml"/> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8178.xml"/>
    <xi:include xmlns:xi="http://www.w3.org/2001/XInclude"
       href="https://xml2rfc.tools.ietf.org/public/rfc/bibxml/reference.RFC.8881.xml"/>
  </references> href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8881.xml"/>
</references>

<section numbered="true" removeInRFC="false" numbered="false" toc="default">
      <name>Acknowledgments</name>
      <t>
        Trond Myklebust, Tom Haynes, and David Flynn
      <t><contact fullname="Trond Myklebust"/>, <contact fullname="Tom
      Haynes"/>, and <contact fullname="David Flynn"/> all worked on the
      prototype at Hammerspace.
      </t>
      <t>
        Dave Noveck, Chuck Lever, Rick Macklem, and Zaheduzzaman Sarker Hammerspace.</t>
      <t><contact fullname="Dave Noveck"/>, <contact fullname="Chuck Lever"/>,
      <contact fullname="Rick Macklem"/>, and <contact fullname="Zaheduzzaman
      Sarker"/> provided reviews of the document.
      </t>
      <t>
        Jeff Layton document.</t>
      <t><contact fullname="Jeff Layton"/> provided experience from an
      implementation he authored.
      </t> authored.</t>
    </section>

</back>

<!-- [rfced] We updated the sourcecode type in Sections 2.1, 3.1, and 5.2 to
"xdr". Please confirm that this is correct.

Should the sourcecode type be set for the sourcecode in Sections 4.1 and 6?

Note that the current list of preferred values for "type" is available at
<https://www.rfc-editor.org/rpc/wiki/doku.php?id=sourcecode-types>.
If the current list does not contain an applicable type, feel free to
suggest additions for consideration. Note that it is also acceptable
to leave the "type" attribute not set.
-->

<!-- [rfced] Please review the "Inclusive Language" portion of the online
Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language>
and let us know if any changes are needed.  Updates of this nature typically
result in more precise language, which is helpful for readers.

For example, please consider whether "white space" should be updated in the
text below:

Original:
   The effect of the script is to remove leading white space from each
   line, plus a sentinel sequence of "///".
-->

</rfc>