INTERNET-DRAFT T. Davis Expires: 15 Feb 1999 Tekelec <draft-davis-xdr-ext-01.txt> August 1998 XDR Extensions Status of this Memo This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." To view the entire list of current Internet-Drafts, please check the "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). ABSTRACT This document describes extensions to the External Data Representation Standard (XDR) protocol as it is currently deployed and accepted. Davis [Page 1]
INTERNET-DRAFT XDR Extensions 29 August 1998 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . 3 2. XDR Data Types . . . . . . . . . . . . . . . . . . . . . 3 2.1 Discriminated Union . . . . . . . . . . . . . . . . . . . 3 2.2 Bit Fields . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Raw Integers . . . . . . . . . . . . . . . . . . . . . . . 6 3. Discussion . . . . . . . . . . . . . . . . . . . . . . . 6 4. The XDR Language Specification . . . . . . . . . . . . . 8 4.1 Syntax Information . . . . . . . . . . . . . . . . . . . 8 5. Bit Field Implementation Example . . . . . . . . . . . . 10 6. References . . . . . . . . . . . . . . . . . . . . . . . 13 7. Security Considerations . . . . . . . . . . . . . . . . . 13 8. Author's Address . . . . . . . . . . . . . . . . . . . . 13 Davis [Page 2]
INTERNET-DRAFT XDR Extensions 29 August 1998 1. Introduction The XDR is a standard for the description and encoding of data in a machine architecture independent form. Full details of the XDR standard can be found in RFC 1832 [1]. It is the focus of this memo to recommend enhancements to the existing specification [1]. This memo addresses three primary issues: (1) The limitations of the discriminated union in representing existing message protocols. (2) The unresolved issue of representing bit fields in a space efficient manor. (3) The representation of byte oriented integers. The overall intention of this memo is to improve the existing standard for use with a broader scope of applications. All assumptions and constraints implied in [1] still remain implied in this memo. 2. XDR Data Types 2.1 Discriminated Union As described in RFC 1832 [1], a discriminated union is a type composed of a discriminant followed by a type selected from a set of prearranged types according to the value of the discriminant. The type of discriminant is either "int", "unsigned int", or an enumerated type, such as "bool". The component types are called "arms" of the union, and are preceded by the value of the discriminant which implies their encoding. However, different from [1] is addition of the structure component declarations which if any must precede the first "arm" declaration and the optional union identifier declaration. The difference is small and therefore allows the interpretation of the union type described in [1] without change. Discriminated unions are declared as follows: union switch (discriminant-declaration, union-identifier) { component-declaration-A; component-declaration-B; ... case discriminant-value-A: arm-declaration-A; Davis [Page 3]
INTERNET-DRAFT XDR Extensions 29 August 1998 case discriminant-value-B: arm-declaration-B; ... default: default-declaration; } identifier; Each "case" keyword is followed by a legal value of the discriminant. The default arm is optional. If it is not specified, then a valid encoding of the union cannot take on unspecified discriminant values. The size of the implied arm is always a multiple of four bytes. The discriminated union is encoded as its discriminant followed by the encoding of the implied arm if no components are specified. 0 1 2 3 +---+---+---+---+---+---+---+---+ | discriminant | implied arm | DISCRIMINATED UNION +---+---+---+---+---+---+---+---+ |<---4 bytes--->| If any components are specified then the components follow the discriminant and precede the implied arm in order as if they were a sub-structure. 0 1 2 3 +---+---+---+---+---+---+---+---+---+---+---+---+ | discriminant | components | implied arm | +---+---+---+---+---+---+---+---+---+---+---+---+ |<---4 bytes--->| The second form provides for cleaner interpretation on addition data when translated into programming languages. For example, in the C programming language the data could be represented by the following structure. struct { discriminant-declaration; component-declaration-A; component-declaration-B; ... union { arm-declaration-A; arm-declaration-B; ... } union-identifier; } identifier; The union-identifier is not part of the encoded data stream and is Davis [Page 4]
INTERNET-DRAFT XDR Extensions 29 August 1998 therefore optional. It is provided as an aid to pre-compilers. 2.2 Bit Fields A XDR bit field is a special case of a signed or unsigned integer from 1 to 32 bits wide. Bit fields can only exist inside a XDR structure or union; however, they can be mixed inside the same structure or union with non-bit field types. The only part of a union where bit fields are permitted is the structure like component section not the union arms. Bit fields are described as follows: int identifier : n; unsigned int identifier : n; where the constant n indicates the width from 1 to 32 bits. Solo bit fields are packed to 32 bits wide starting with the most significant bit. All remaining bits (padding) are zero filled. +---------+---------+---------+---------+---------+ | Bit n-1 | ... | Bit 1 | Bit 0 | Padding + BIT FIELD +---------+---------+---------+---------+---------+ <---------------------- 32 Bits ------------------> Consecutive bit fields are packed into 32 bit blocks. A bit field is never split across the 32 bit blocks. In this case, the previous block is zero padded to 32 bits and the bit field starts in the next 32 bit block. The order of consecutive bit fields is determined be the order of declaration as are all other XDR types. For example: struct { int i : 4; int j : 8; unsigned int k : 16; int l : 16; int m : 4; }; 0 4 12 28 32/0 16 20 31 +---+-------+---------------+---+---------------+---+-----------+ | i | j | k | | l | m | | +---+-------+---------------+---+---------------+---+-----------+ <----- Block 0, 32 Bits ------> <----- Block 1, 32 Bits ------> <-------------------------- 64 Bits --------------------------> The term "consecutive" refers to adjacent bit fields in the same Davis [Page 5]
INTERNET-DRAFT XDR Extensions 29 August 1998 structure or union. 2.3 Raw Integers A XDR raw integer is a special case of a signed or unsigned integer. They are very much like the opaque data type but fixed to a length of 4 bytes. Unlike the opaque data type these are represented as an integer instead of a byte array. For example, an IP address is often coded as an integer but is really 4 bytes. The numeric value of the IP address may change on different machines but the byte order does not. Raw integers are descibed as follows: raw int identifier; raw unsigned int identifier; Raw integers are encoded with the left most byte first, MSB on big endian and LSB if little endian. This is exactly the same as the raw data reguardless of machine. Endian ------ Big MSB LSB Little LSB MSB +--------+--------+--------+--------+ | Byte 0 | Byte 1 | Byte 2 | Byte 3 + RAW INTEGER +--------+--------+--------+--------+ <--------------- 32 Bits -----------> 3. Discussion (1) Why change the discriminated union? The proposed type allows for easier application with existing protocols. For example, many protocols place the message type in the first 4 bytes of each message and it is common place for that type to be followed by other data common the all messages of that protocol in a header. The union data in this event usually follows this header. RFC 1832 [1] does not directly handle this situation. However, it can be shown that if the encoding does not follow the order of the non-encoded data members the same effect can be reached with the previous XDR discriminated union. Secondly, if existing protocols need to transfer machine independent data, having the machine independent data be able to model existing data will make the transition less costly. While it is not possible to seamlessly convert existing data types to a XDR syntax, it is Davis [Page 6]
INTERNET-DRAFT XDR Extensions 29 August 1998 possible to reassemble existing data into XDR format and seamlessly convert it into most language formats by a pre-compiler. The more closely the XDR syntax can resemble existing data types the easier the transition of existing applications will be. (2) Why add the bit field data type? Although the bit fields are little more than integers. They represent a common part of programming languages today. Perhaps the most common use of bit fields is to describe data in a packed format for efficiency. It is a good idea to allow that description and purpose to be passed along in a machine independent style. Consider the situation in [1], if 32 separately but consecutively described bit fields of width 1 are encoded, exactly 128 bytes in the encoded stream are used. Where in this style exactly 4 bytes are used. That is a 99.97% reduction in the encoded data stream for the best case and a 0% reduction in the worst case. (3) Why add the raw integer data type? Consider an application that tranfers IP addresses from one host to another. Also consider that the application keeps a list of IP addresses as a series of longs. The IP address is not really a number at all, it is a series of bytes that is stored easily as a long integer. On different machines the numeric value of the number may change because the byte order does not. Now, how does this application send the data in the XDR stream? If it sends this data as a integer then the byte order may change making the IP addresses backwards. The only choice is to send it as a 4 byte opaque. However, this confuses the interpretation of it in the application. Naturally, the application could consider it a union for type casting but that makes it pretty ugly and adds 4 more bytes in the encoded XDR stream. The application developer needs flexibilty in how he or she wants to represent the data. This type provides a great deal of flexibilty without complexity. Davis [Page 7]
INTERNET-DRAFT XDR Extensions 29 August 1998 4. The XDR Language Specification Only the "Syntax Information" section of part of the language specification has changed and is there for the only part included in this memo. Are other parts from [1] still apply without correction. 4.1 Syntax Information The following diagram includes the diagram in RFC 1832 [1] for completeness. The same lexical and syntax notes apply from [1]. declaration: type-specifier identifier | type-specifier identifier "[" value "]" | type-specifier identifier "<" [ value ] ">" | "opaque" identifier "[" value "]" | "opaque" identifier "<" [ value ] ">" | "string" identifier "<" [ value ] ">" | type-specifier "*" identifier | "void" bitfield-inclusive-declaration: [ "unsigned" ] "int" identifier ":" value | declaration value: constant | identifier type-specifier: [ "raw" ] [ "unsigned" ] "int" | [ "unsigned" ] "hyper" | "float" | "double" | "quadruple" | "bool" | enum-type-spec | struct-type-spec | union-type-spec | identifier enum-type-spec: "enum" enum-body enum-body: "{" ( identifier "=" value ) Davis [Page 8]
INTERNET-DRAFT XDR Extensions 29 August 1998 ( "," identifier "=" value )* "}" struct-type-spec: "struct" struct-body struct-body: "{" ( bitfield-inclusive-declaration ";" ) ( bitfield-inclusive-declaration ";" )* "}" union-type-spec: "union" union-body union-body: "switch" "(" declaration [ "," identifier ] ")" "{" ( bitfield-inclusive-declaration ";" )* ( "case" value ":" declaration ";" ) ( "case" value ":" declaration ";" )* [ "default" ":" declaration ";" ] "}" constant-def: "const" identifier "=" constant ";" type-def: "typedef" declaration ";" | "enum" identifier enum-body ";" | "struct" identifier struct-body ";" | "union" identifier union-body ";" definition: type-def | constant-def specification: definition * Davis [Page 9]
INTERNET-DRAFT XDR Extensions 29 August 1998 5. Bit Field Implementation Example The following code segment illustrates one possible implementation of bit field packing and unpacking of the XDR stream. The style implied follows the currently implemented style of XDR coding. This function returns FALSE on failure and TRUE on success. This function does not take the actual structure members as its arguments because machine independent packing of the bits may vary. It is the assumption of the function that a complete list of the "consecutive" bit fields are provided in the arguments. /* * The following define simply masks out bits that should not * be present the a current bitfield. */ #define MASK_WIDTH(value, width) ( \ ((u_long)0xffffffff) >> ( 32 - (width) ) \ & ((u_long)(value)) \ ) bool_t xdr_bitfields( register XDR *xdr, /* XDR stream pointer */ u_int num_fields, /* Number of bit fields */ u_char *widths, /* Bit field widths */ u_long *values /* Bit field values */ ) { u_long val; /* Temporary storage of packed bit fields */ u_int i; /* Loop counter */ i_int offset; /* Current offset in 32-bit block */ if ( num_fields <= 0 ) return TRUE; switch ( xdrs->x_op ) { case XDR_ENCODE: /* * ENCODING: For each bit field, if there is Davis [Page 10]
INTERNET-DRAFT XDR Extensions 29 August 1998 * room in the current 32-bit block, pack it; * otherwise, encode the current block and pack * the bit field into the next block. * * When all bit fields are packed encode the final * 32-bit block. */ val = 0; offset = 0; for ( i = 0; i < num_fields; i++ ) { if ( widths[i] < 1 || widths[i] > 32 ) return ERROR; if ( offset + widths[i] > 32 ) { if ( xdr_u_long( xdrs, &val ) == FALSE ) return FALSE; offset = 0; val = 0; } offset += widths[i]; val |= ( MASK_WIDTH( values[i], widths[i] ) \ << ( 32 - offset ) ); } if ( xdr_u_long( xdrs, &val ) == FALSE ) return FALSE; return TRUE; case XDR_DECODE: /* * DECODING: Decode the first block. For each * bit field, if there is enough space in the * current 32-bit block, unpack it; otherwise, * decode the next block and unpack the bit * field from it. */ Davis [Page 11]
INTERNET-DRAFT XDR Extensions 29 August 1998 if ( xdr_u_long( xdrs, &val ) == FALSE ) return FALSE; offset = 0; for ( i = 0; i < num_fields; i++ ) { if ( widths[i] < 1 || widths[i] > 32 ) return ERROR; if ( offset + widths[i] > 32 ) { if ( xdr_u_long( xdrs, &val ) == FALSE ) return FALSE; offset = 0; } offset += widths[i]; values[i] = MASK_WIDTH( val >> (32 - offset), \ widths[i]); } return TRUE; case XDR_FREE: return TRUE; } return FALSE; } Davis [Page 12]
INTERNET-DRAFT XDR Extensions 29 August 1998 6. References [1] Srinivasan, R., "XDR: External Data Representation Standard", STD 1, RFC 1832, Sun Microsystems, Inc., August 1995. 7. Security Considerations Security issues are not discussed in this memo. 8. Author's Address Thomas Davis Tekelec, Inc. 5151 McCrimmon Parkway Suite 216 Morrisville, NC 27560 Phone: 919-380-2066 Fax: 919-380-2060 EMail: tom.davis@tekelec.com <draft-davis-xdr-ext-01.txt> Expires: 15 Feb 1999 Davis [Page 13]