NFSv4 WG 11-08-2012 BEEPY: NetApp did a late IPA disclosure around FedFS work. Find it on IETF web site. Very permissive use of the patent not only for NFSv4. Give it as a Xmas present! Please use it. The lesson is: If you are aware of any patent or potential patent, disclosure early and often. BEEPY: Spencer, it's about 6:05AM your time, if you're not on line I'm pissed! ---------- Updates. (Tom Haynes) 3530bis: Frank Filz from IBM had some issues. Martin? no comment. FedFS protocol work: Chuck will talk about it. IESG actions: Brian? BRIAN: report> Chuck FedFS IESG responses. Labeled NFS: informational - under Martin bis document: Tom just reported. .x12 on xdr representation 4.2 work: 2-3 issues left with consensus, wrap up on work, expect no problem's RPCSEC_GSSv3: Tom will address in slides NFSv4 migration, implementation and spec issues: Tom will address. ----------- FedFS NSDB and ADMIN updates. (Chuck L) Three docs. dns srv rfc 6641 nothing to do admin protocol, and nsdb protocol in IESG review Since Vancouver: admin protocol already in last call, Chuck made editor for IESG review. IANA and Ballot review addressed in repository Work with Martin later to finalize telechat on 15 November FedFS protocol (NSDB) Significant changes since Vancouver NSDB schema to LDAP community. Chuck fixed problems, updated draft, prototyped on linux successfully. Went into last call on OCT 8. LDAP OID expert approved. telechat on 15 Nov. MARTIN: shift forward two weeks? Chuck has no problems. Ballot discussion on NSDB draft. MAY vrs MUST - should be addressed in GitHub. IANA. two new registries. IANA not clear about request details. Where to create? Under LDAP under NFSv4 registry....etc. Chuck requests FedFS parameters as new top level registry. BOAZ: why separate from LDAP parameters? CHUCK: because using existing OID Ongoing FedFS work - multidomain - maintaining standards documents - support for SMB FSLs IEGS process issue: no LDAP expert. What do we do next time? MARTIN: we did ask for an early review - response was they had no time, but we did ask way in advance CHUCK: still an open question. MARTIN: not sure. depends on other ADs. TOM: spencer: - what is the intent of SMB work? standards track? CHUCK: Write an informational draft and see - there will be an ietf product. MARTIN: LDAP issue - assigned us Lief J. as technical guide to do the LDAP review. You can shame him into action because he is listed on the charter! ----------- NFSv4.2 (Tom Haynes) All content added. need some design review, including minorversioning language. Minoversioning: intent is that we don't turn a feature off. pull in minorversion rules into 4.2 doc and clarify. CHUCK: why not move minorversioning rules into their own document? instead of updating in various versions... BRIAN: has no opinion: list as open issue. what are the variations and how are they changing. BENNY: what are the current rules and why are they changing? TOM : what can change in a minorversion is the issue. We have READ?WRTIE and want to mark them as obsolete in the next version ... but then we may not fit the spirit of the minorversion is to do this - but the current rule says we must say they are obsolete right now. BEN: deprecated? TOM: Yes add deprecation into new minor version rules. discuss on mailing list. PRANOOP: fix the original version - don't do it in a new document, do it in 5661 MARTIN:SPENCER: discuss on mailinig list. Serverside Copy : renamed ops, added stateid for lock recovery. need someone to review stateid - will ask Trond. WRITE-PLUS: renamed. async text to use ss-copy offload. biggest text change need to review. SPENCER: when are the design review meetings: TOM: start in two weeks, use slot we have reserved. SPENCER: that's thanksgiving TOM: OK! ------------ RPCSEC_GSSv3. (Tom Haynes) -04 not changes. annotated for AI's Nico's comments that need to be detailed close to wrapping up. Dependencies: Labeled NFS, Server side copy, multi-domain access Linux implementation. any other? no. Are we going to continue with this document? MATT: linux-box: What are the trade offs TOM: allows you to send a subject (labeled NFS) SPENCER: have to keep working on this else 4.2 stops. ANDY: I'll help review and finalize. ------------ Multi-domain FedFS (Andy A.) see slides. ------------ pNFS Lustre layout discussion. (Sorin Faibish) SORIN: target Linux implementation. Motivation: Lustre uses it's own "RDMA" data protocol to DS. will be lustre client in the kernel ANDROS: if lustre client in linux kernel, why need pNFS layout in Linux? SORIN: using pNFS should improve performance BENNY: Standardization process needs two implementations. how does one implement this without spec. of lustre protocol? proper IETF draft for protocol should be a prerequisite for this layout. SORIN: based on an existing protocol BANNY: trying to capture the lustre client protocol in IETF document so there is a formal protocol definition SORIN: will be a document that will define protocol that will be stable and won't change. IETF doc, fine. BENNY: doesn't have to be IETF - block is like this, but need document. SORIN: OK will do. MARTIN: similar issue. routing group to use rsync to manage state - no spec. impossible to normative refer... in the end they wrote something up. needs to be accessible, complete enough to implement, stable. BOAZ: by end of year 2012, lustre patches submitted for review?. SORIN: Yes. Intel and EMC will support the implementation for lustre client in linux kernel. EMC will implement the pNFS client and the pNFS MDS server for Linux. Will relasx lustre POSIX consistency to NFS close-to-open consistency. BENNY: comment- not sure all clustered apps will be happy with close-to-open consistency SORIN: will be optional - still just a proposal. BENNY: imagine a clustered scientific app rw sharing mode, what would it use to keep striped cache consistency? SORIN: using mpio so consistency is weak . locking issues in Lustre are problematic, they don't use open/close.... trying to address. BENNY: basically saying mpio - which requires no protocol support SORIN: specfs 2013 - big problem mounting lustre and getting good results. BOAZ: client is in kernel, then all apps can just use the Lustre client, new platforms can use new pNFS open-to-close.... SORIN: yes. question of acceptance. See why it is needed slide. draft reviewed by D.Black, J.Glascow, T.Hayes. BEN: Freebsd have no chance of implementing Lustre client without a spec. BOAZ: async journal commit mech (AJCM): NFS protocol and linux NFS client already has an AJCM. duplication of AJCM is a waste. NFS client is already doing server reboot, reclaim, error recovery - all of the page state etc are AJCM. Two pointsr: 1) waste of two mech for the same job. 2) looks like it will break because lustre client will do one thing, NFS will try to do another.... SORIN:just trying to improve what lustre is bringing to the table BOAZ: AJCM should not be in the layout driver. making LD stronger and smarter than it needs to be is a problem use the NFS AJMC SORIN: OK - will consider as we write draft will bring this issue out. You are right and it can possibly be solved. BOAZ: last comment. weird still referencing a moving target protocol. said documentation of protocol will be part of the source control together with content. On the wire protocol will change every major release. SORIN: other protocols are the same BOAZ: maybe will change versions - that is different. You need source control, and protocol definition control. SORIN: yes. ------------ pNFS NFS - objects layout (Benny H) yet anoter new layout proposal. adds NFS as storage access protocol - MDS as a security manager. hands out capabilities per cred to allow access to data server. adds NFS as storage protocol and as a back end protocol, as T-10 is used in 5664. Motivation: NFS is ubiquitous. T-10 deployment is scarce. - aggregate stand alone data servers into a cluster. - export exiting existing clustered file systems e.g. Ceph, Gluster. SORIN: Lustre implements pNFS on top of Lustre. BENNY: Yes, but Luster requires another layer of indirection - no direct access. SORIN: this is the motivation for Lustre layout BENNY: we may be able to merge efforts if Lustre uses NFS as data BOAZ: not true: Just run Lustre client on client - local host gateway. BENNY: flexible per file layout is why Ceph, Gluster cannot use file layout. BENNY: MDS owns security hands out per-file rpc creds as a RO RW capability. capablity gates use of NFS access by changing ownership of file to fence. (see slides) TOM: this is an undocumented process. Lustre wasn't documented. T-10 capability protocol for capabilities is not documented. BENNY: initial draft document it, but I agree MIKE E: wouldn't it be simpler to just use NFS existing security ACL, real security BENNY: looking for fencing off client solution. MIKE: revoke the file handle BENNY that is an undocumented feature TOM: it is with v3 - if using v4 just muck with the stateid. BOAZ: files layout back end protocol not defined, just requirements. fence off stateids. Benny wants to define that back end protocol. MIKE: Back end protocol is required to use this layout? BENNY: the way it is now, no. but would like to propose a standard MKIE: In absence of this protocol, the security is not up to par with NFSv4.x. so why is this standards track? BENNY: i don't think NFSv4 replaced NFSv3 based on security requirement. v3 is used even though v4 is there. MIKE: netapp has a lot of customers - interested in security v4 acls and krb5. using lustre where data protocol is underspecified or v3 - no standards body that has control over v3 - oracle owns it, so basing a layout on it is a non -starter TOM: don't like to have a definition of a back end protocol. stick with the requirements of back end. do it for v4. v3 is a non-starter. MATT: core features of this draft are unique. mix-match are key. objective of flexible per file striping may be one for further discussion. meaningful for file layout as well. MATT: separate from these drafts, there is a motivation for a files layout that is capable of flexible per file layouts. PRANOOP: abstraction of devices and layout is what matt is talking about. one device and multiple layouts is problem. opportunity to fix that in the files layout. MATT: yes. BOAZ: disagree completely. a per file layout as opposed to a per device layout is already the object layout 5664. there is no doubt that all the things you need can go under the third option benny's proposed. file layout - per device, object - per file. so don't repeat. BENNY: if there is an opportunity to reduce # of layouts - good MIKE: not opposed to support idea of MDS DS protocol TOM: just dont want it mandated. We should be open to extending the files layout. new or Benny's. object layout does not meet the security needs. BOAZ: where does t-10 lack security? BENNY: mandatory byte range locking. because T-10 does not know stateid, so can't enforce lock BOAS: each capabilty has a range. so MDS can revoke all the capabilities that use that range - access denied to get a new layout, so wait for the lock. BENNY: note - that is not documented in 5664. BOAZ: nothing to document. Same as RAID 5 where you can't write to the same stripe from two owners. SPENCER: do we decide for an update to files layout? if so, need requirements. TOM: we are talking about multiple groups to update the file layout. do we define a standard back end protocol? BENNY: do both. define requirements then new doc or update file. PRANOOP: agreed. this is an issue. BOAZ: benny wants RAID 5/6 recalls...already exists. why a new file layout? nothing new - in objects. Benny is trying to do is the marriage between the per-files layout state management V4.1 want to reuse. why a new one? Benny's proposal is not new. BOAZ: My proposal is that with a small change to Bennys proposal we can add to 5664 instead of a 4th layout type that needs to be added the the enum in 5661. lets keep 5664 the major document not duplicate any text, but make Benny's doc a new OSD transport definition. BENNY not technically possible. BOAZ: keep component structure the same as today with single difference : capability nfs v4 string is variable length (in 5664) and client must keep it the same as received by the MDS. THe object layout map and component ra all stay the same. now have capability that is per transport. Most comments benny got on his document are things missing in 5664. Just define a different transport version for 5664. OSD-NFS transport. BENNY: Boaz, please dive into the details in a document. On mailing list. BOAZ: do people like the idea of new transport. SORIN: Spencer has right direction. if big change in layout or data protocol deserves a new draft. like the idea of expanding existing drafts. BOAZ: i have a real concrete solution. 5661 deferred to per layout the device/file layout management. I still say use 5664.... BRIAN: not sure I'm hearing correctly. is Benny making changes to his doc to respond to comments - or does there need to be some other effort to make a draft in response. BOAZ: last question -standard body procedures. is it not easier to add another new draft describing a transport vrs a layout? BRIAN: i don't have an answer to that. PRANOOP: confused - what are we trying to solve. BOAZ: benny - new layout type. he could not do that, keep 5664, just have another transport defined. EMMERSON: nfsv41 has a mech to allow what types it supports (layout) no infrastructure to enumerate transports from server before getting layout. TOM: not working group that decides it - put the effort out to complete BRIAN: modulo a completely different approach - help Benny! if completely different counter proposal, then write it up and put it on the table. BRIAN: Chuck - you have a comment on this? CHUCK: No, something new BRIAN: Chuck - remember, I have your memory card as hostage! CHUCK: curious to know about implementations of session vrs clientid trunking... informal poll. EMERSON: for NFS Ganesha, implemented preliminary version of session trunking BRIAN: check the mailing list - get a feel for progress... ------------ BRIAN: On a final note - Remember, if filing a patent - Martin and i would like you to announce it! MARTION: If you don't disclose an IPR you will be whacked!