You are here: irt.org | RFCs | RFC707 [ previous next ]
NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing THE GOAL, RESOURCE SHARING 1 The principal goal of all resource-sharing computer networks, including the now international ARPA Network (the ARPANET), is to usefully interconnect geographically distributed hardware, software, and human resources [1]. Achieving this goal requires the design and implementation of various levels of support software within each constituent computer, and the specification of network-wide "protocols" (that is, conventions regarding the format and the relative timing of network messages) governing their interaction. This paper outlines an alternative to the approach that ARPANET system builders have been taking since work in this area began in 1970, and suggests a strategy for modeling distributed systems within any large computer network. 1a The first section of this paper describes the prevailing ARPANET protocol strategy, which involves specifying a family of application-dependent protocols with a network-wide inter-process communication facility as their common foundation. In the second section, the application-independent command/response discipline that characterizes this protocol family is identified and its isolation as a separate protocol proposed. Such isolation would reduce the work of the applications programmer by allowing the software that implements the protocol to be factored out of each applications program and supplied as a single, installation-maintained module. The final section of this paper proposes an extensible model for this class of network interaction that in itself would even further encourage the use of network resources. 1b -1- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing The Current Software Approach to Resource Sharing THE CURRENT SOFTWARE APPROACH TO RESOURCE SHARING 2 Function-Oriented Protocols 2a The current ARPANET software approach to facilitating resource sharing has been detailed elsewhere in the literature [2, 3, 4]. Briefly, it involves defining a Host-Host Protocol by which the operating systems of the various "host" computers cooperate to support a network-wide inter-process communication (IPC) facility, and then various function-oriented protocols by which processes deliver and receive specific services via IPC. Each function-oriented protocol regulates the dialog between a resident "server process" providing the service, and a "user process" seeking the service on behalf of a user (the terms "user" and "user process" will be used consistently throughout this paper to distinguish the human user from the computer process acting on his behalf). 2a1 The current Host-Host Protocol has been in service since 1970. Since its initial design and implementation, a variety of deficiencies have been recognized and several alternative protocols suggested [5, 6]. Although improvements at this level would surely have a positive effect upon Network resource sharing, the present paper simply assumes the existence of some form of IPC and focuses attention upon higher level protocol design issues. 2a2 Each of the function-oriented protocols mentioned in this paper constitutes the official ARPANET protocol for its respective application domain and is therefore implemented at nearly all of the 75 host installations that now comprise the Network. It is primarily upon this widely implemented protocol family (and the philosophy it represents) that the present paper focuses. Needless to say, other important resource sharing tools have also been constructed within the ARPANET. The Resource Sharing Executive (RSEXEC), designed and implemented by Bolt, Beranek and Newman, Inc [7], provides an excellent example of such work. 2a3 Experience with and Limitations of Hands-On Resource Sharing 2b The oldest and still by far the most heavily used function-oriented protocol is the Telecommunications Network protocol (TELNET) [8], which effectively attaches a terminal on one computer to an interactive time-sharing system on another, and allows a user to interact with the remote system via the terminal as if he were one of its local users. 2b1 -2- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing The Current Software Approach to Resource Sharing As depicted in Figure 1, TELNET specifies the means by which a user process monitoring the user's terminal is interconnected, via an IPC communication channel, with a server process with access to the target time-sharing system. TELNET also legislates a standard character set in which the user's commands and the system's responses are to be represented in transmission between machines. The syntax and semantics of these interchanges, however, vary from one system to another and are unregulated by the protocol; the user and server processes simply shuttle characters between the human user and the target system. 2b2 Although the hands-on use of remote resources that TELNET makes possible is a natural and highly visible form of resource sharing, several limitations severely reduce its long-term utility: 2b3 (1) It forces upon the user all of the trappings of the resource's own system. To exploit a remote resource, the user must leave the familiar working environment provided by his local system and enter an alien one with its own peculiar system structure (login, logout, and subsystem entry and exit procedures) and command language discipline (command recognition and completion conventions, editing characters, and so on). Hands-on resource sharing thus fails to provide the user with the kind of organized and consistent workshop he requires to work effectively [9]. (2) It provides no basis for bootstrapping new composite resources from existing ones. Because the network access discipline imposed by each resource is a human-engineered command language, rather than a machine-oriented communication protocol, it is virtually impossible for one resource to programatically draw upon the services of others. Doing so would require that the program deal successfully with complicated echoing and feedback characteristics; unstructured, even unsolicited system responses; and so forth. Hands-on resource sharing thus does nothing to provide an environment in which existing resources can be used as building blocks to construct new, more powerful ones. These inherent limitations of hands-on resource sharing are removed by a protocol that simplifies and standardizes the dialog between user and server processes. Given such a protocol, the -3- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing The Current Software Approach to Resource Sharing various remote resources upon which a user might wish to draw can indeed be made to appear as a single, coherent workshop by interposing between him and them a command language interpreter that transforms his commands into the appropriate protocol utterances [10, 11]. The construction of composite resources also becomes feasible, since each resource is accessible by means of a machine-oriented protocol and can thus be readily employed by other processes within the network. 2b4 Standardizing the Inter-Machine Dialog in Specific Application Areas 2c After the TELNET protocol had been designed and widely implemented within the ARPANET, work began on a family of function-oriented protocols designed for use by programs, rather than human users. Each such protocol standardizes the inter-machine dialog in a particular application area. While TELNET dictates only the manner in which user and server processes are interconnected via the IPC facility, and the character set in which the two processes communicate once connected, each member of this family specifies in addition the syntax and semantics of the commands and responses that comprise their dialog. 2c1 Protocols within this family necessarily differ in substance, each specifying its own application-specific command set. The File Transfer Protocol (FTP) [12], for example, specifies commands for manipulating files, and the Remote Job Entry Protocol (RJE) [13] specifies commands for manipulating batch jobs. Protocols throughout the family are, however, similar in form, each successive family member having simply inherited the physical features of its predecessors. Thus FTP and RJE enforce the same conventions for formulating commands and responses. 2c2 This common command/response discipline requires that commands and responses have the following respective formats: 2c3 command-name <SP> parameter <CRLF> response-number <SP> text <CRLF> Each command invoked by the user process is identified by NAME and is allowed a single PARAMETER. Each response generated by the server process contains a three-digit decimal response NUMBER (to be interpreted by the user process) and explanatory TEXT (for presentation, if necessary, to the user). Response numbers are assigned in such a way that, for example, positive and negative acknowledgments can be easily distinguished by the user process. 2c4 -4- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing The Current Software Approach to Resource Sharing FTP contains, among others, the following commands (each listed with one of its possible responses) for retrieving, appending to, replacing, and deleting files, respectively, within the server process' file system: 2c5 Command Response RETR <SP> filename <CRLF> 250 <SP> Beginning transfer. <CRLF> APPE <SP> filename <CRLF> 400 <SP> Not implemented. <CRLF> STOR <SP> filename <CRLF> 453 <SP> Directory overflow. <CRLF> DELE <SP> filename <CRLF> 450 <SP> File not found. <CRLF> The first three commands serve only to initiate the transfer of a file from one machine to another. The transfer itself occurs on a separate IPC channel and is governed by what amounts to a separate protocol. 2c6 Since the general command format admits but a single parameter, multiparameter operations must be implemented as sequences of commands. Thus two commands are required to rename a file: 2c7 Command Response RNFR <SP> oldname <CRLF> 200 <SP> Next parameter. <CRLF> RNTO <SP> newname <CRLF> 253 <SP> File renamed. <CRLF> -5- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach A COMMAND/RESPONSE PROTOCOL, THE BASIS FOR AN ALTERNATIVE APPROACH 3 The Importance of Factoring Out the Command/Response Discipline 3a That FTP, RJE, and the other protocols within this family share a common command/response discipline is a fact not formally recognized within the protocol literature, and each new protocol document describes it in detail, as if for the first time. Nowhere are these conventions codified in isolation from the various contexts in which they find use, being viewed as a necessary but relatively unimportant facet of each function-oriented protocol. "This common command/response discipline has thus gone unrecognized as the important, application-independent protocol that it is." 3a1 This oversight has had two important negative effects upon the growth of resource sharing within the ARPANET: 3a2 (1) It has allowed the command/response discipline to remain crude. As already noted, operations that require more than a single parameter are consistently implemented as two or more separate commands, each of which requires a response and thus incurs the overhead of a full round-trip network delay. Furthermore, there are no standards for encoding parameter types other than character strings, nor is there provision for returning results in a command response. (2) It has placed upon the applications programmer the burden of implementing the network "run-time environment (RTE)" that enables him to access remote processes at the desired, functional level. Before he can address remote processes in terms like the following: execute function DELE with argument TEXTFILE on machine X the applications programmer must first construct (as he invariably does in every program he writes) a module that provides the desired program interface while implementing the agreed upon command/response discipline. This run-time environment contains the code required to properly format outgoing commands, to interface with the IPC facility, and to parse incoming responses. Because the system provides only -6- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach the IPC facility as a foundation, the applications programmer is deterred from using remote resources by the amount of specialized knowledge and software that must first be acquired. If, on the other hand, the command/response discipline were formalized as a separate protocol, its use in subsequent function-oriented protocols could rightly be anticipated by the systems programmer, and a single run-time environment constructed for use throughout an installation (in the worst case, one implementation per programming language per machine might be required). This module could then be placed in a library and, as depicted in Figure 2, link loaded with (or otherwise made available to) each new applications program, thereby greatly simplifying its use of remote resources. Furthermore, since enhancements to it would pay dividends to every applications program employing its services, the run-time environment would gradually be augmented to provide additional new services to the programmer. The thesis of the present paper is that one of the keys to facilitating network resource sharing lies in (1) isolating as a separate protocol the command/response discipline common to a large class of applications protocols; (2) making this new, application-independent protocol flexible and efficient; and (3) constructing at each installation a RTE that employs it to give the applications programmer easy and high-level access to remote resources. 3a3 Specifications for the Command/Response Protocol 3b Having argued the value of a command/response protocol (hereafter termed the Protocol) as the foundation for a large class of applications protocols, there remains the task of suggesting the form that the Protocol might take. There are eight requirements. First, it must reproduce the capabilities of the discipline it replaces: 3b1 (1) Permit invocation of arbitrary, named commands (or functions) implemented by the remote process. (2) Permit command outcomes to be reported in a way that aids both the program invoking the commmand and the user under whose control it may be executing. -7- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach Second, the Protocol should remove the known deficiencies of its predecessor, that is: 3b2 (3) Allow an arbitrary number of parameters to be supplied as arguments to a single command. (4) Provide representations for a variety of parameter types, including but not limited to character strings. (5) Permit commands to return parameters as results as well as accept them as arguments. And, finally, the Protocol should provide whatever additional capabilities are required by the more complex distributed systems whose creation the Protocol seeks to encourage. Although others may later be identified, the three capabilities below are recognized now to be important: 3b3 (6) Permit the server process to invoke commands in the user process, that is, eliminate entirely the often inappropriate user/server distinction, and allow each process to invoke commands in the other. In the workshop environment alluded to earlier, for example, the user process is the command language interpreter and the server process is any of the software tools available to the user. While most commands are issued by the interpreter and addressed to the tool, occasionally the tool must invoke commands in the interpreter or in another tool. A graphical text editor, for example, must invoke commands within the interpreter to update the user's display screen after an editing operation. (7) Permit a process to accept two or more commands for concurrrent execution. The text editor may wish to permit the user to initiate a long formatting operation with one command and yet continue to issue additional, shorter commands before there is a response to the first. (8) Allow the process issuing a command to suppress the response the command would otherwise elicit. This feature would permit network traffic to be reduced in those cases in which the process invoking the command deems a -8- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach response unnecessary. Commands that always succeed but never return results are obvious candidates for this kind of treatment. A Formulation of the Protocol That Meets These Specifications 3c The eight requirements listed above are met by a protocol in which the following two messages are defined: 3c1 message-type=COMMAND [tid] command-name arguments message-type=RESPONSE tid outcome results Here and in subsequent protocol descriptions, elements enclosed in square brackets are optional. 3c2 The first message invokes the command whose NAME is specified using the ARGUMENTS provided. The second is issued in eventual response to the first and returns the OUTCOME and RESULTS of the completed command. Whenever OUTCOME indicates that a command has failed, the command's RESULTS are required to be an error number and diagnostic message, the former to help the invoking program determine what to do next, the latter for possible presentation to the user. The protocol thus provides a framework for reporting errors, while leaving to the applications program the tasks of assigning error numbers and composing the text of error messages. 3c3 There are several elements of the Protocol that are absent from the existing command/response discipline: 3c4 (1) RESULTS, in fulfillment of Requirement 5. (2) A MESSAGE TYPE that distinguishes commands from responses, arising from Requirement 6. In the existing discipline, this distinction is implicit, since user and server processes receive only responses and commands, respectively. (3) An optional transaction identifier TID by which a command and its response are associated, arising from Requirements 7 and 8. The presence of a transaction identifier in a command implies the necessity of a response echoing the identifier; and no two concurrently outstanding commands may bear the same identifier. -9- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach Requirements 3 and 4--the ability to transmit an arbitrary number of parameters of various types with each command or response--are most economically and effectively met by defining a small set of primitive "data types" (for example, booleans, integers, character strings) from which concrete parameters can be modeled, and a "transmission format" in which such parameters can be encoded.Appendix A suggests a set of data types suitable for a large class of applications; Appendix B defines some possible transmission formats. 3c5 The protocol description given above is, of course, purely symbolic. Appendix C explores one possible encoding of the Protocol in detail. 3c6 Summarizing the Arguments Advanced So Far 3d The author trusts that little of what has been presented thus far will be considered controversial by the reader. The following principal arguments have been made: 3d1 (1) The more effective forms of resource sharing depend upon remote resources being usefully accessible to other programs, not just to human users. (2) Application-dependent protocols providing such access using the current approach leave to the applications programmer the task of constructing the additional layer of software (above the IPC facility provided by the system) required to make remote resources accessible at the functional level, thus discouraging their use. (3) A single, resource-independent protocol providing flexible and efficient access at the functional level to arbitrary remote resources can be devised. (4) This protocol would make possible the construction at each installation of an application-independent, network run-time environment making remote resources accessible at the functional level and thus encouraging their use by the applications programmer. A protocol as simple as that suggested here has great potential for stimulating the sharing of resources within a computer network. First, it would reduce the cost of adapting existing resources for network use by eliminating the need for the design, documentation, and implementation of specialized delivery protocols. Second, it -10- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A Command/Response Protocol, the Basis for an Alternative Approach would encourage the use of remote resources by eliminating the need for application-specific interface software. And finally, it would encourage the construction of new resources built expressly for remote access, because of the ease with which they could be offered and used within the network software marketplace. 3d2 -11- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A High-Level Model of the Network Environment A HIGH-LEVEL MODEL OF THE NETWORK ENVIRONMENT 4 The Importance of the Model Imposed by the Protocol 4a The Protocol proposed above imposes upon the applications programmer a particular model of the network environment. In a heterogeneous computer network, nearly every protocol intended for general implementation has this effect, since it idealizes a class of operations that have concrete but slightly different equivalents in each system. Thus the ARPANET's TELNET Protocol alluded to earlier, for example, specifies a Network Virtual Terminal that attempts to provide a best fit to the many real terminals in use around the Network. 4a1 As now formulated, the Protocol models a remote resource as an interactive program with a simple, rigidly specified command language. This model follows naturally from the fact that the function-oriented protocols from which the Protocol was extracted were necessitated by the complexity and diversity of user-oriented command languages. The Protocol may thus legitimately be viewed as a vehicle for providing, as an adjunct to the sophisticated command languages already available to users, a family of simple command languages that can readily be employed by programs. 4a2 While the command/response model is a natural one, others are possible. A remote resource might also be modeled as a process that services and replies to requests it receives from other computer processes. This request/reply model would emphasize the fact that the Protocol is a vehicle for inter-process communication and that no human user is directly involved. 4a3 Substituting the request/reply model for the command/response model requires only cosmetic changes to the Protocol: 4a4 message-type=REQUEST [tid] op-code arguments message-type=REPLY tid outcome results In the formulation above, the terms "REQUEST", "REPLY", and "op-code" have simply been substituted for "COMMAND", "RESPONSE", and "command-name", respectively. 4a5 The choice of model need affect neither the content of the Protocol nor the behavior of the processes whose dialog it governs. Use of the word "command" in the command/response model, for example, is not meant to imply that the remote process can be coerced into action. Whatever model is adopted, a process has -12- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A High-Level Model of the Network Environment complete freedom to reject an incoming remote request that it is incapable of or unwilling to fulfill. 4a6 But even though it has no substantive effect upon the Protocol, the selection of a model--command/response, request/reply, and so on--is an important task because it determines the way in which both applications and systems programmers perceive the network environment. If the network environment is made to appear foreign to him, the applications programmer may be discouraged from using it. The choice of model also constrains the kind and range of protocol extensions that are likely to occur to the systems programmer; one model may suggest a rich set of useful extensions, another lead nowhere (or worse still, in the wrong direction). 4a7 In this final section of the paper, the author suggests a network model (hereafter termed the Model) that he believes will both encourage the use of remote resources by the applications programmer and suggest to the systems programmer a wide variety of useful Protocol extensions. Unlike the substance of the Protocol, however, the Model has already proven quite controversial within the ARPANET community. 4a8 Modeling Resources As Collections of Procedures 4b Ideally, the goal of both the Protocol and its accompanying RTE is to make remote resources as easy to use as local ones. Since local resources usually take the form of resident and/or library subroutines, the possibility of modeling remote commands as "procedures" immediately suggests itself. The Model is further confirmed by the similarity that exists between local procedures and the remote commands to which the Protocol provides access. Both carry out arbitrarily complex, named operations on behalf of the requesting program (the caller); are governed by arguments supplied by the caller; and return to it results that reflect the outcome of the operation. The procedure call model thus acknowledges that, in a network environment, programs must sometimes call subroutines in machines other than their own. 4b1 Like the request/reply model already described, the procedure call model requires only cosmetic changes to the Protocol: 4b2 message-type=CALL [tid] procedure-name arguments message-type=RETURN tid outcome results In this third formulation, the terms "CALL", "RETURN", and "procedure-name" have been substituted for "COMMAND, "RESPONSE", and -13- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A High-Level Model of the Network Environment "command-name", respectively. And in this form, the Protocol might aptly be designated a "procedure call protocol (PCP)". 4b3 "The procedure call model would elevate the task of creating applications protocols to that of defining procedures and their calling sequences. It would also provide the foundation for a true distributed programming system (DPS) that encourages and facilitates the work of the applications programmer by gracefully extending the local programming environment, via the RTE, to embrace modules on other machines." This integration of local and network programming environments can even be carried as far as modifying compilers to provide minor variants of their normal procedure-calling constructs for addressing remote procedures (for which calls to the appropriate RTE primitives would be dropped out). 4b4 Finally, the Model is one that can be naturally extended in a variety of ways (for example, coroutine linkages and signals) to further enhance the distributed programming environment. 4b5 Clarifying the Procedure Call Model 4c Although in many ways it accurately portrays the class of network interactions with which this paper deals, the Model suggested above may in other respects tend to mislead the applications programmer. The Model must therefore be clarified: 4c1 (1) Local procedure calls are cheap; remote procedure calls are not. Local procedure calls are often effected by means of a single machine instruction and are therefore relatively inexpensive. Remote procedure calls, on the other hand, would be effected by means of a primitive provided by the local RTE and require an exchange of messages via IPC. Because of this cost differential, the applications programmer must exercise discretion in his use of remote resources, even though the mechanics of their use will have been greatly simplified by the RTE. Like virtual memory, the procedure call model offers great convenience, and therefore power, in exchange for reasonable alertness to the possibilities of abuse. (2) Conventional programs usually have a single locus of control; distributed programs need not. -14- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing A High-Level Model of the Network Environment Conventional programs are usually implemented as a single process with exactly one locus of control. A procedure call, therefore, traditionally implies a transfer of control from caller to callee. Distributed systems, on the other hand, are implemented as two or more processes, each of which is capable of independent execution. In this new environment, a remote procedure call need not suspend the caller, which is capable of continuing execution in parallel with the called procedure. The RTE can therefore be expected to provide, for convenience, two modes of remote procedure invocation: a blocking mode that suspends the caller until the procedure returns; and a non-blocking mode that releases the caller as soon as the CALL message has been sent or queued. Most conventional operating systems already provide such a mode choice for I/O operations. For non-blocking calls, the RTE must also, of course, either arrange to asynchronously notify the program when the call is complete, or provide an additional primitive by which the applications program can periodically test for that condition. Finally, the applications programmer must recognize that by no means all useful forms of network communication are effectively modeled as procedure calls. The lower level IPC facility that remains directly accessible to him must therefore be employed in those applications for which the procedure call model is inappropriate and RTE-provided primitives simply will not do. 4c2 -15- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Some Expectations SOME EXPECTATIONS 5 Both the Procedure Call Protocol and its associated Run-Time Environment have great potential for facilitating the work of the network programmer; only a small percentage of that potential has been discussed in the present paper. Upon the foundation provided by PCP can be erected higher level application-independent protocol layers that further enhance the distributed programming environment by providing even more powerful capabilities (see Appendix D). 5a As the importance of the RTE becomes fully evident, additional tasks will gradually be assigned to it, including perhaps those of: 5b (1) Converting parameters between the format employed internally by the applications program, and that imposed by the Protocol. 5b1 (2) Automatically selecting the most appropriate inter-process transmission format on the basis of the two machines' word sizes. 5b2 (3) Automatically substituting for network IPC a more efficient form of communication when both processes reside on the same machine. 5b3 The RTE will eventually offer the programmer a wide variety of application-independent, network-programming conveniences, and so, by means of the Protocol, become an increasingly powerful distributed-system-building tool. 5c -16- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Acknowledgments ACKNOWLEDGMENTS 6 Many individuals within both SRI's Augmentation Research Center (ARC) and the larger ARPANET community have contributed their time and ideas to the development of the Protocol and Model described in this paper. The contributions of the following individuals are expressly acknowledged: Dick Watson, Jon Postel, Charles Irby, Ken Victor, Dave Maynard, and Larry Garlick of ARC; and Bob Thomas and Rick Schantz of Bolt, Beranek and Newman, Inc. 6a ARC has been working toward a high-level framework for network-based distributed systems for a number of years now [14]. The particular Protocol and Model described here result from research begun by ARC in July of 1974. This research included developing the Model; designing and documenting the Protocol required to support it [15]; and designing, documenting, and implementing a prototype run-time environment for a particular machine [16, 17], specifically a PDP-10 running the Tenex operating system developed by Bolt, Beranek and Newman, Inc [18]. Three design iterations were carried out during a 12-month period, and the resulting specification implemented for Tenex. The Tenex RTE provides a superset of the capabilities presented in the body of this paper and Appendices A through C as well as those alluded to in Appendix D. 6b The work reported here was supported by the Advanced Research Projects Agency of the Department of Defense, and by the Rome Air Development Center of the Air Force. 6c -17- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix A: Suggested Data Types APPENDIX A: SUGGESTED DATA TYPES 7 The Protocol requires that every parameter or "data object" be represented by one of several primitive data types defined by the Model. The set of data types below is sufficient to conveniently model a large class of data objects, but since the need for additional data types (for example, floating-point numbers) will surely arise, the set must remain open-ended. Throughout the descriptions below, N is confined to the range [0, 2**15-1]: 7a LIST: A list is an ordered sequence of N data objects called "elements". A LIST may contain other LISTs as elements, and can therefore be employed to construct arbitrarily complex composite data objects. 7a1 CHARSTR: A character string is an ordered sequence of N ASCII characters, and conveniently models a variety of textual entities, from short user names to whole paragraphs of text. 7a2 BITSTR: A bit string is an ordered sequence of N bits and, therefore, provides a means for representing arbitrary binary data (for example, the contents of a word of memory). 7a3 INTEGER: An integer is a fixed-point number in the range [-2**31, 2**31-1], and conveniently models various kinds of numerical data, including time intervals, distances, and so on. 7a4 INDEX: An index is an integer in the range [1, 2**15-1]. As its name and value range suggest, an INDEX can be used to address a particular bit or character within a string, or element within a list. INDEXes have other uses as well, including the modeling of handles or identifiers for open files, created processes, and the like. Also, because of their restricted range, INDEXes are more compact in transmission than INTEGERs (see Appendix B). 7a5 BOOLEAN: A boolean represents a single bit of information, and has either the value true or false. 7a6 EMPTY: An empty is a valueless place holder within a LIST or parameter list. 7a7 -18- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix B: Suggested Transmission Formats APPENDIX B: SUGGESTED TRANSMISSION FORMATS 8 Parameters must be encoded in a standard transmission format before they can be sent from one process to another via the Protocol. An effective strategy is to define several formats and select the most appropriate one at run-time, adding to the Protocol a mechanism for format negotiation. Format negotiation would be another responsibility of the RTE and could thus be made completely invisible to the applications program. 8a Suggested below are two transmission formats. The first is a 36-bit binary format for use between 36-bit machines, the second an 8-bit binary, "universal" format for use between dissimilar machines. Data objects are fully typed in each format to enable the RTE to automatically decode and internalize incoming parameters should it be desired to provide this service to the applications program. 8b PCPB36, For Use Between 36-Bit Machines 8c Bits 0-13 Unused (zero) 8c1 Bits 14-17 Data type 8c2 EMPTY =1 INTEGER=4 LIST=7 BOOLEAN=2 BITSTR =5 INDEX =3 CHARSTR=6 Bits 18-20 Unused (zero) 8c3 Bits 21-35 Value or length N 8c4 EMPTY unused (zero) BOOLEAN 14 zero-bits + 1-bit value (TRUE=1/FALSE=0) INDEX unsigned value INTEGER unused (zero) BITSTR unsigned bit count N CHARSTR unsigned character count N LIST unsigned element count N Bits 36- Value 8c5 EMPTY unused (nonexistent) BOOLEAN unused (nonexistent) INDEX unused (nonexistent) INTEGER two's complement full-word value BITSTR bit string + zero padding to word boundary CHARSTR ASCII string + zero padding to word boundary LIST element data objects -19- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix B: Suggested Transmission Formats PCPB8, For Use Between Dissimilar Machines 8d Byte 0 Data type 8d1 EMPTY =1 INTEGER=4 LIST=7 BOOLEAN=2 BITSTR =5 INDEX =3 CHARSTR=6 Bytes 1- Value 8d2 EMPTY unused (nonexistent) BOOLEAN 7 zero-bits + 1-bit value (TRUE=1/FALSE=0) INDEX 2-byte unsigned value INTEGER 4-byte two's complement value BITSTR 2-byte unsigned bit count N + bit string + zero padding to byte boundary CHARSTR 2-byte unsigned character count N + ASCII string LIST 2-byte element count N + element data objects -20- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix C: A Detailed Encoding of the Procedure Call Protocol APPENDIX C: A DETAILED ENCODING OF THE PROCEDURE CALL PROTOCOL 9 Although the data types and transmission formats detailed in the previous appendixes serve primarily as vehicles for representing the arguments and results of remote procedures, they can just as readily and effectively be employed to represent the commands and responses by which those parameters are transmitted. 9a Taking this approach, one might model each of the two Protocol messages as a PCP data object, specifically a LIST whose first element is an INDEX message type. The following concise statement of the Protocol then results: 9b LIST (CALL, tid, procedure, arguments) INDEX=1 INDEX/EMPTY CHARSTR LIST 9b1 LIST (RETURN, tid, outcome, results) INDEX=2 INDEX BOOLEAN LIST 9b2 The RESULTS of an unsuccessful procedure would be represented as follows: 9c LIST (error, diagnostic) INDEX CHARSTR 9c1 -21- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix D: A Look at Some Possible Extensions to the Model APPENDIX D: A LOOK AT SOME POSSIBLE EXTENSIONS TO THE MODEL 10 The result of the distributed-system-building strategy proposed in the body of this paper and the preceeding appendices is depicted in Figure D-1. At the core of each process is the inter-process communication facility provided by the operating system, which effects the transmission of arbitrary binary data between distant processes. Surrounding this core are conventions regarding first the format in which a few, primitive types of data objects are encoded in binary for IPC, and then the formats of several composite data objects (that is, messages) whose transmission either invokes or acknowledges the previous invocation of a remote procedure. Immediately above lies an open-ended protocol layer in which an arbitrary number of enhancements to the distributed programming environment can be implemented. Encapsulating these various protocol layers is the installation-provided run-time environment, which delivers DPS services to the applications program according to machine- and possibly programming-language-dependent conventions. 10a The Protocol proposed in the present paper recognizes only the most fundamental aspects of remote procedure calling. It permits the caller to identify the procedure to be called, supply the necessary arguments, determine the outcome of the procedure, and recover its results. In a second paper [19], the author proposes some extensions to this simple procedure call model, and attempts to identify other common forms of inter-process interaction whose standardization would enhance the distributed programming environment. Included among the topics discussed are: 10b (1) Coroutine linkages and other forms of communication between the caller and callee. 10b1 (2) Propagation of notices and requests up the thread of control that results from nested procedure calls. 10b2 (3) Standard mechanisms for remotely reading or writing system-global data objects within another program. 10b3 (4) Access controls for collections of related procedures. 10b4 (5) A standard means for creating and initializing processes, that is, for establishing contact with and logging into a remote machine, identifying the program to be executed, and so forth. This facility would permit arbitrarily complex process hierarchies to be created. 10b5 -22- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Appendix D: A Look at Some Possible Extensions to the Model (6) A mechanism for introducing processes to one another, that is, for superimposing more general communication paths upon the process hierarchy. 10b6 These and other extensions can all find a place in the open-ended protocol layer of Figure D-1. The particular extensions explored in [19] are offered not as dogma but rather as a means of suggesting the possibilities and stimulating further research. 10c -23- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing References REFERENCES 11 1. Kahn, R. E., "Resource-Sharing Computer Communications Networks," Proceedings of the IEEE, Vol. 60, No. 11, pp. 1397-1407, November 1972. 11a 2. Crocker, S. D., Heafner, J. F., Metcalfe, R. M., Postel, J. B., "Function-oriented Protocols for the ARPA Computer Network," AFIPS Proceedings, Spring Joint Computer Conference, Vol. 40, pp. 271-279, 1972. 11b 3. Carr, C. S., Crocker, S. D., Cerf, V. G., "Host-Host Communication Protocol in the ARPA Network," AFIPS Proceedings, Spring Joint Computer Conference, Vol. 36, pp. 589-597, 1970. 11c 4. Mc Kenzie, A. A., Host/Host Protocol for the ARPA Network, Bolt Beranek and Newman Inc., Cambridge, Massachusetts, January 1972 (SRI-ARC Catalog Item 8246). 11d 5. Walden, D. C., "A System for Interprocess Communication in a Resource Sharing Computer Network," Communications of the ACM, Vol. 15, No. 4, pp. 221-230, April 1972. 11e 6. Cerf, V. G., Kahn, R. E., "A Protocol for Packet Network Intercommunication," IEEE Transactions on Communications, Vol. Com-22, No. 5, pp. 637-648, May 1974. 11f 7. Thomas, R. H., "A Resource-Sharing Executive for the ARPANET," AFIPS Proceedings, National Computer Conference, Vol. 42, pp. 155-163, 1973. 11g 8. TELNET Protocol Specification, Stanford Research Institute, Menlo Park, California, August 1973 (SRI-ARC Catalog Item 18639). 11h 9. Engelbart, D. C., Watson, R. W., Norton, J. C., "The Augmented Knowledge Workshop," AFIPS Proceedings, National Computer Conference, Vol. 42, pp. 9-21, 1973. 11i 10. Engelbart, D. C., English, W. K., "A Research Center for Augmenting Human Intellect," AFIPS Proceedings, Fall Joint Computer Conference, Vol. 33, pp. 395-410, 1968. 11j 11. Irby, C. H., Dornbush, C. F., Victor, K. E., Wallace, D. C., "A Command Meta Language for NLS," Final Report, Contract -24- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing References RADC-TR-75-304, SRI Project 1868, Stanford Research Institute, Menlo Park, California, December, 1975. 11k 12. Neigus, N. J., File Transfer Protocol, ARPA Network Working Group Request for Comments 542, Bolt Beranek and Newman Inc., Cambridge, Massachusetts, July 1973 (SRI-ARC Catalog Item 17759). 11l 13. Bressler, R. D., Guida, R., Mc Kenzie, A. A., Remote Job Entry Protocol, ARPA Network Working Group Request for Comments 360, Dynamic Modeling Group, Massachusetts Institute of Technology, Cambridge, Massachusetts, (undated) (SRI-ARC Catalog Item 12112). 11m 14. Watson, R. W., Some Thoughts on System Design to Facilitate Resource Sharing, ARPA Network Working Group Request for Comments 592, Augmentation Research Center, Stanford Research Institute, Menlo Park, California, November 20, 1973 (SRI-ARC Catalog Item 20391). 11n 15. White, J. E., DPS-10 Version 2.5 Implementer's Guide, Augmentation Research Center, Stanford Research Institute, Menlo Park, California, August 15, 1975 (SRI-ARC Catalog Item 26282). 11o 16. White, J. E., DPS-10 Version 2.5 Programmer's Guide, Augmentation Research Center, Stanford Research Institute, Menlo Park, California, August 13, 1975 (SRI-ARC Catalog Item 26271). 11p 17. White, J. E., DPS-10 Version 2.5 Source Code, Augmentation Research Center, Stanford Research Institute, Menlo Park, California, August 13, 1975 (SRI-ARC Catalog Item 26267). 11q 18. Bobrow, D. G., Burchfiel, J. D., Murphy, D. L., Tomlinson, R. S., "TENEX, a Paged Time Sharing System for the PDP-10," Communications of the ACM, Vol. 15, No. 3, pp. 135-143, March 1972. 11r 19. White, J. E., "Elements of a Distributed Programming System," Submitted for publication in the Journal of Computer Languages, 1976. 11s -25- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 NCC 76 A High-Level Framework for Network-Based Resource Sharing Figure List FIGURE LIST 12 Figure 1. Interfacing a remote terminal to a local time-sharing system via the TELNET Protocol. 12a Figure 2. Interfacing distant applications programs via their run-time environments. 12b Figure D-1. Software and protocol layers comprising a process within the distributed programming system. 12c -26- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 -27- NWG/RFC# 707 JEW 14-JAN-76 19:51 34263 A High-Level Framework for Network-Based Resource Sharing 23-DEC-75 James E. White Augmentation Research Center Stanford Research Institute Menlo Park, California 94025 (415) 326-6200 x2960 This paper proposes a high-level, application-independent protocol and software framework that would extend the local programming environment to embrace modules in other computers within a resource sharing computer network, and thereby facilitate the construction of distributed systems and encourage the sharing of resources. The work reported here was supported by the Advanced Research Projects Agency of the Department of Defense, and by the Rome Air Development Center of the Air Force. This paper has been submitted for publication in the Proceedings of the 1976 National Computer Conference.