NFSv4 Working Group S. Faibish
Internet-Draft EMC Corporation
Intended status: draft J. Glasgow
Expires: September 7, 2011 Google
Updates: 5663 D. Black
EMC Corporation
March 7, 2011
pNFS block disk protection
draft-faibish-nfsv4-pnfs-block-disk-protection-00
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on September 7, 2011.
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
Faibish et al. Expires September 7, 2011 [Page 1]
Internet-Draft pNFS block disk protection March 2011
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
Abstract
Parallel NFS (pNFS) extends Network File Sharing version 4 (NFSv4) to
allow client to directly access file data on the storage used by the
NFSv4 server. This ability to bypass the server for data access can
increase both performance and parallelism, but requires additional
client functionality for data access, some of which is dependent of
the class of storage used. The published protocol for block layout
file systems describes how clients can recognize the volumes used for
pNFS storage, but only after communicating with the server. This
document proposes a mechanism by which clients can recognize block
storage devices used by pNFS file systems, without having to
communicate with the server, and therefore allows the clients to
limit access to the devices at client boot time.
Table of Contents
1. Introduction...................................................3
1.1. Non-pNFS clients use case.................................5
1.2. pNFS client use case......................................5
2. Conventions used in this document..............................5
3. Extending block/volume signature...............................5
3.1. Modifications to 2.3.5 RFC5663............................7
4. Problem Statement..............................................7
4.1. Non-pNFS clients..........................................7
4.2. Solution..................................................7
4.3. pNFS clients..............................................8
5. Changes to GETDEVICEINFO and or LAYOUTCOMMIT (RFC5661).........8
6. Security Considerations........................................8
7. IANA Considerations............................................9
8. Conclusions....................................................9
9. References.....................................................9
9.1. Normative References......................................9
9.2. Informative References....................................9
Authors Addresses...............................................10
Faibish et al. Expires September 7, 2011 [Page 2]
Internet-Draft pNFS block disk protection March 2011
1. Introduction
Figure 1 shows the overall architecture of a Parallel NFS (pNFS)
system:
+-----------+
|+-----------+ +-----------+
||+-----------+ | |
||| | NFSv4.1 + pNFS | |
+|| Clients |<------------------------------>| MDS |
+| | | |
+-----------+ | |
||| +-----------+
||| |
||| |
||| Storage +-----------+ |
||| Protocol |+-----------+ |
||+----------------||+-----------+ Control |
|+-----------------||| | Protocol |
+------------------+|| Storage |------------+
+| Devices |
+-----------+
Figure 1 pNFS Architecture
In this document, "storage device" is used as a general term for a
data server and/or storage server for the file, block or object pNFS
layouts.
In the current pNFS block protocol [RFC5663] the client has to
contact the MDS in order to get the signature offset of the devices
used by the pNFS block client.
If an operating system wants to be able to identify a device/LUN
being used to store pNFS data during the boot sequence, before the
mount is issued, without having to contact the metadata server it is
not possible because the protocol does not define a fixed location
for a signature or for an identifier of a device as being used by
pNFS. The MDS will send the device ID after mount time. In order for
the OS to mount it needs to have the network configured and the IP
address of the pNFS server and this always happens after the
discovery of SCSI devices.
Faibish et al. Expires September 7, 2011 [Page 3]
Internet-Draft pNFS block disk protection March 2011
location of the signature is also not defined in the protocol but
we can make it a configuration parameter in the client OS that will
define the signature location for each pNFS block server, vendor
neutral. The preferred solution would be to enhance the block
protocol to include a signature unique identifying pNFS. It needs to
be included in the protocol as both clients and server need to know
this information.
In more details the pNFS FS writes a signature to the LUN, but to
find out the offset of the signature the client first needs to talk
with the MDS to be told the offset that the signature is located at
using GETDEVICEINFO. But this is too late and can only be done after
the boot ends. An OS that wants to hide LUNs from users that contain
pNFS data even if that host doesn't mount the pNFS block volume
cannot do this.
The problem is that there is a window of time before an OS
utility/daemon can be started and allow to protect/prevent to write
to the pNFS devices and the time when kernel application with higher
priority than the above daemon can overwrite the pNFS devices. As a
result of this it may be possible that some kernel application will
write over the pNFS FS devices/LUNs. This is only a pNFS block issue
and will have no secondary effect on OS kernel operation.
An OS will check the signature of pNFS should be able to identify the
devices and prevent from writing to them. But this enhancement should
allow such a protection mechanism to be implemented. This is outside
the scope of this draft to detect and protect the devices if the
specific pNFS signature is detected and do so without the need to
contact the MDS "if there is pNFS specific component of the
signature. Even in this case we can use generic identifiers or some
opaque in the protocol can recognize pNFS devices even though pNFS
block protocol doesn't require any protection.
Typically, storage area network (SAN) disk arrays and SAN protocols
provide access control mechanisms (e.g., Logical Unit Number (LUN)
mapping and/or masking), which operate at the granularity of
individual hosts, not individual blocks. For this reason, block-
based protection must be provided by the client software.
Since block/volume storage systems are generally not capable of
enforcing such file-based security, in environments where pNFS
clients cannot be trusted to enforce such policies, pNFS block/volume
storage layouts SHOULD NOT be used.
Faibish et al. Expires September 7, 2011 [Page 4]
Internet-Draft pNFS block disk protection March 2011
1.1. Non-pNFS clients use case
The non-pNFS clients will need to check the signature related to pNFS
block devices and prevent from WRITE data to the devices that have
the pNFS identifier. The non-pNFS client OS MAY decide to implement
an application that will read the pNFS signature at a known location
(LBA) on the block device and write protect the device. Or it MAY
just detect the pNFS devices and prevent using them for other
applications that use block devices. The current draft only make
these use cases possible and non-pNFS clients MAY NOT use this
feature.
1.2. pNFS client use case
pNFS clients MAY use this signature in cases when a pNFS MDS server
sends layout extents to devices that were erroneously not signed
using this pNFS identification by the server and included in valid
layouts. The pNFS client MAY check the pNFS signature for devices
that are included in valid layouts and report an error to the MDS in
the LAYOUTCOMMIT or using a permission access mechanism similar to
[PAC]. This use case is relevant in cases when virtual LUs are used
as pNFS devices and they are added after the client issued a
GETDEVICEINFO. Alternatively when the pNFS client detects the missing
signature it may send a GETDEVICEINFO command to the MDS for the
faulty device and use a permission error code to the MDS.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [RFC2119].
3. Extending block/volume signature
The pNFS block layout identifies storage volumes by content, for
example providing the means to match (unique portions of) labels used
by volume managers. Volume identification is performed by matching
one or more opaque byte sequences to specific parts of the stored
data. Any block pNFS system using this layout MUST support a means
of content-based unique volume identification that can be employed
via the data structure given here. The location of the pNFS signature
will be defined in the remaining of the document.
Faibish et al. Expires September 7, 2011 [Page 5]
Internet-Draft pNFS block disk protection March 2011
The GUID used in the GPT should not be confused with the device
signature described in section 2.2.1 of [RFC5663]. The GUID is the
same for all pNFS volumes where as the signature composed of
pnfs_block_sig_component4 components is unique to each pNFS volume.
The location on disk of these identifiers are unrelated.
A pNFS signature is an array of up to "PNFS_BLOCK_MAX_DISK_SIG_COMP"
(defined below) signature components. The client MUST NOT assume that
all signature components are co-located within a single sector on a
block device. The pNFS client block layout driver uses this volume
identification to identify block devices used by pNFS file systems.
The pNFS device signature may require several LBAs in order to ensure
uniqueness of the signature in case that other disk signatures exist
on the same location on the block device and prevent confusions with
other signatures.
All the new pNFS identifiers must not be used in extents covering
data access and need to be read only accessible to both pNFS and
non- pNFS clients. Additional, pNFS clients SHOULD NOT access/write
to volumes that do not have the pNFS specific volume identifier if
included in a valid layout.
This can be relevant in cases of virtual LUs when they are extended
to include new volumes uninitialized by the server. After the
GETDEVICEINFO is received by the client the client MAY read the pNFS
identifier in order to validate the extents on that device and
prevent from writing to devices that do not include pNFS identifier.
In this case pNFS clients will send an error to the server and inform
it that the layout is invalid and or that there is a permission
access error. And will fallback the I/O to the MDS. The error
mechanism can be similar to the one used in the [pNFS-draft-
permission-access] for lack of permission to access reinforced by the
pNFS client.
pNFS client implementations MAY chose not to validate that the device
included in the layouts received from the server is a pNFS device.
The pNFS client can OPTIONALLY check the pNFS identifier and send an
error message in the layoutcommit for any device with a missing pNFS
identifier.
Faibish et al. Expires September 7, 2011 [Page 6]
Internet-Draft pNFS block disk protection March 2011
3.1. Modifications to 2.3.5 RFC5663
The above assumption that extents are permissions may be in
contradiction to the case when a device in the layout has no pNFS
identity without the knowledge of the pNFS server. This because the
pNFS client is the first to detect the lack of identifier while the
server is unaware of this issue and assumed that the device used in
the layout and the respective extent has the right permissions. The
server MAY check that the device used in the layout is a legitimate
pNFS device at the time it respond to the GETDEVICEINFO call from the
client.
Alternatively in the case when a device has no pNFS identifier the
pNFS client SHOULD send an GETDEVICEINFO for that device to ensure
that the device included in a valid layout can be accessed for IO. It
may or may not report an error in the GETDEVICEINFO and/or in the
LAYOUTCOMMIT at the end of the write I/O.
The check can be made for both read-only layouts but SHOULD be done
in the case of read-write layouts.
The server is responsible to ensure that the pNFS id is written to
the devices used by the pNFS FSID before allowing pNFS clients to
mount the fsid.
4. Problem Statement
4.1. Non-pNFS clients
The pNFS block layout [RFC5663] requires that storage systems allow
both the server and multiple clients to access block storage devices.
In principle, access control to block storage devices can be achieved
via LUN masking techniques at the storage server as volumes are
mounted. In practice this is not always done, and storage devices
are often accessible independent of whether or not a client has
mounted the associated pNFS file system. For this reason, it is
desirable to enable clients to identify pNFS block storage in order
to prevent non-pNFS access by the client.
4.2. Solution
The pNFS block layout specification allows a client to identify the
storage devices associated with a mounted volume via the
GETDEVICELIST and GETDEVICEINFO operations. These operations can
only be issued once a client has established contact with the server,
and thus do not allow a client to block non-pNFS access to pNFS
storage devices in all cases. For environments in which it is
Faibish et al. Expires September 7, 2011 [Page 7]
Internet-Draft pNFS block disk protection March 2011
desirable for clients to block non-pNFS access to pNFS block storage,
the following SHOULD be done:
- Each storage device dedicated to pNFS includes a GUID partition
table (GPT).
- The pNFS Block Storage partitions are identified in the GPT with
GUID e5b72a69-23e5-4b4d-b176-16532674fc34.
- NFS clients do not issue NFS operations for non-pNFS access to
any storage identified as pNFS Block Storage by that GUID.
This enables an NFS client to prevent non-pNFS access to pNFS block
storage immediately upon boot. As of 2010 most current OSs support
GPT including FreeBSD, Linux and Solaris [GPT]. Block/volume storage
is logically at a lower layer of the I/O stack than the network stack
and hence during the client boot cycle access to block devices is
possible before NFSv4 security can be enforced. The client MUST
prevent WRITE I/Os to devices that will be later identified as pNFS
devices. It is the responsibility of those administering and
deploying pNFS with a block/volume storage access protocol to ensure
that appropriate protection is provided to pNFS devices identifiable
by the client. This protection is similar to the mechanism that
protect GPT tables written on the block devices used by different
operating systems.
4.3. pNFS clients
In this case the client MAY check if the devices in the GETDEVICEINFO
list have pNFS signature at mount time. At I/O time the pNFS client
MAY check the signature to validate that the block devices are valid
and report to the server the error.
5. Changes to GETDEVICEINFO and or LAYOUTCOMMIT (RFC5661)
[Need to include the changes if we agree on the need for this]
6. Security Considerations
The security considerations of 2.6 [RFC5663] apply to this document.
This document formalizes a mechanism which allows client operating
systems to limit access to pNFS block storage devices. By doing so
it allows for increase security in environments in which the client
operating system is trusted. As with the issues discussed in the
security section of 2.6 [RFC5663], in environments where the security
Faibish et al. Expires September 7, 2011 [Page 8]
Internet-Draft pNFS block disk protection March 2011
requirements are such that client-side protection from access to
storage is not sufficient, pNFS block/volume storage layouts SHOULD
NOT be used. Furthermore, in such environments, SAN mechanism (e.g.,
LUN mapping and/or masking) should be used to limit access.
7. IANA Considerations
There are no IANA considerations in this document beyond pNFS IANA
Considerations are covered in [RFC5661].
8. Conclusions
This draft specifies additions to the pNFS block protocol addressing
protection of disks used by pNFS clients for non-pNFS clients access.
9. References
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 Protocol",
http://tools.ietf.org/html/rfc5661, January 2010.
[RFC5663] Black, D., Glasgow, J., Fridella, S., "Parallel NFS (pNFS)
Block/Volume Layout", http://tools.ietf.org/html/rfc5663,
January 2010.
[RFC5664] Halevy, B., Welch, B., Zelenka, J., "Object-Based Parallel
NFS (pNFS) Operations", http://tools.ietf.org/html/rfc5664,
January 2010
[XDR] Eisler, M., "XDR: External Data Representation Standard",
STD 67, RFC 4506, May 2006.
9.2. Informative References
[GPT] http://en.wikipedia.org/wiki/GUID_Partition_Table
[PAC] https://datatracker.ietf.org/doc/draft-ietf-nfsv4-pnfs-
access-permissions-check/
This document was prepared using 2-Word-v2.0.template.dot.
Faibish et al. Expires September 7, 2011 [Page 9]
Internet-Draft pNFS block disk protection March 2011
Authors Addresses
Sorin Faibish (editor)
EMC Corporation
228 South Street
Hopkinton, MA 01748
US
Phone: +1 (508) 305-8545
Email: sfaibish@emc.com
Jason Glasgow
Google
5 Cambridge Center, Floors 3-6
Cambridge, MA 02142
US
Phone: +1 (617) 575 1599
Email: jglasgow@google.com
David L. Black
EMC Corporation
176 South Street
Hopkinton, MA 01748
US
Phone: +1 (508) 293-7953
Email: david.black@emc.com
Faibish et al. Expires September 7, 2011 [Page 10]