CRNP
This chapter provides information about the Cluster Reconfiguration Notification Protocol (CRNP). CRNP enables failover and scalable applications to be "cluster aware." More specifically, CRNP provides a mechanism that enables applications to register for, and receive subsequent asynchronous notification of, Sun Cluster reconfiguration events. Data services that run within the cluster and applications that run outside the cluster can register for notification of events. Events are generated when membership in a cluster changes and when the state of a resource group or a resource changes.
Overview of CRNP
CRNP provides mechanisms and daemons that generate cluster reconfiguration events, route them through the cluster, and send them to interested clients.
The cl_apid daemon interacts with the clients. The Sun Cluster Resource Group Manager (RGM) generates cluster reconfiguration events. These daemons use syseventd(1M) to transmit events on each local node. The cl_apid daemon uses Extensible Markup Language (XML) over TCP/IP to communicate with interested clients.
The following diagram presents an overview of the flow of events between the CRNP components. In this diagram, one client is running on cluster node 2, and the other client is running on a computer that is not part of the cluster.
Figure 12-1 How CRNP Works
Overview of the CRNP Protocol
The CRNP defines the Application, Presentation, and Session layers of the standard seven layer Open System Interconnect (OSI) protocol stack. The Transport layer must be TCP and the Network layer must be IP. The CRNP is independent of the Data Link and Physical layers. All Application layer messages that are exchanged in the CRNP are based on XML 1.0.
Semantics of the CRNP Protocol
Clients initiate communication by sending a registration message (SC_CALLBACK_RG) to the server. This registration message specifies the event types for which the clients want to receive notification as well as a port to which the events can be delivered. The source IP of the registration connection and the specified port, taken together, form the callback address.
Whenever an event of interest to a client is generated within the cluster, the server contacts the client on its callback address (IP and port) and delivers the event (SC_EVENT) to the client. The server is highly available, running within the cluster itself. The server stores client registrations in storage that persists even after the cluster is rebooted.
Clients unregister by sending a registration message (SC_CALLBACK_RG, which contains a REMOVE_CLIENT message) to the server. After the client receives an SC_REPLY message from the server, the client closes the connection.
The following diagram shows the flow of communication between a client and a server.
Figure 12-2 Flow of Communication Between a Client and a Server
Message Types That the CRNP Uses
The CRNP uses three types of messages, all of which are XML-based, as described in the following table. These message types are described in more detail later in this chapter. Usage is also described in more detail later in this chapter.
Type of Message | Description |
|---|---|
SC_CALLBACK_REG | This message takes four forms: ADD_CLIENT, REMOVE_CLIENT, ADD_EVENTS, and REMOVE_EVENTS. Each of these forms contains the following information:
The ADD_CLIENT, ADD_EVENTS, and REMOVE_EVENTS forms also contain an unbounded list of event types, each of which includes the following information:
Together, the event class and event subclass define a unique "event type." The DTD (document type definition) from which the classes of SC_CALLBACK_REG are generated is SC_CALLBACK_REG. This DTD is described in more detail in Appendix F, Document Type Definitions for CRNP. |
SC_EVENT | This message contains the following information:
The values in an SC_EVENT are not typed. The DTD (document type definition) from which the classes of SC_EVENT are generated is SC_EVENT. This DTD is described in more detail in Appendix F, Document Type Definitions for CRNP. |
SC_REPLY | This message contains the following information:
The DTD (document type definition) from which the classes of SC_REPLY are generated is SC_REPLY. This DTD is described in more detail in Appendix F, Document Type Definitions for CRNP. |
How a Client Registers With the Server
This section describes how an administrator will set up the server, how clients are identified, how information is sent over the Application and Session layers, and error conditions.
Assumptions About How Administrators Will Set Up the Server
The system administrator must configure the server with a highly available IP address (that is, an IP address that is not tied to a particular machine in the cluster) and a port number. The administrator must publish this network address to prospective clients. The CRNP does not define how this server name is made available to clients. Administrators will either use a naming service, which will enable clients to find the network address of the server dynamically, or will add the network name to a configuration file for the client to read. The server will run within the cluster as a failover resource type.
How the Server Identifies a Client
Each client is uniquely identified by its callback address, that is, its IP address and port number. The port is specified in the SC_CALLBACK_REG messages, and the IP address is obtained from the TCP registration connection. CRNP assumes that subsequent SC_CALLBACK_REG messages with the same callback address come from the same client, even if the source port from which the messages are sent is different.
How SC_CALLBACK_REG Messages Are Passed Between a Client and the Server
A client initiates a registration by opening a TCP connection to the server's IP address and port number. After the TCP connection is established and ready for writing, the client must send its registration message. The registration message must be one correctly formatted SC_CALLBACK_REG message that does not contain extra bytes either before or after the message.
After all the bytes have been written to the stream, the client must keep its connection open to receive the reply from the server. If the client does not format the message correctly, the server does not register the client, and sends an error reply to the client. If the client closes the socket connection before the server sends a reply, the server registers the client as normal.
A client can contact the server at any time. Every time a client contacts the server, the client must send an SC_CALLBACK_REG message. If the server receives a message that is malformed, out of order, or invalid, the server sends an error reply to the client.
A client cannot send an ADD_EVENTS, REMOVE_EVENTS, or REMOVE_CLIENT message before that client sends an ADD_CLIENT message. A client cannot send a REMOVE_CLIENT message before that client sends an ADD_CLIENT message.
If a client sends an ADD_CLIENT message and the client is already registered, the server might tolerate this message. In this situation, the server silently replaces the old client registration with the new client registration that is specified in the second ADD_CLIENT message.
In most situations, a client registers with the server once, when the client starts, by sending an ADD_CLIENT message. And a client unregisters once by sending a REMOVE_CLIENT message to the server. However, the CRNP provides more flexibility for those clients that need to modify their event type list dynamically.
Contents of an SC_CALLBACK_REG Message
Each ADD_CLIENT, ADD_EVENTS, and REMOVE_EVENTS message contains a list of events. The following table describes the event types that the CRNP accepts, including the required name and value pairs.
If a client either:
Sends a REMOVE_EVENTS message that specifies one or more event types for which the client has not previously registered, or
Registers for the same event type twice
Class and Subclass | Name and Value Pairs | Description |
|---|---|---|
EC_Cluster ESC_cluster_membership | Required: none Optional: none | Registers for all cluster membership change events (node death or join) |
EC_Cluster ESC_cluster_rg_state | One required, as follows: rg_name Value type: string Optional: none | Registers for all state change events for resource group name |
EC_Cluster ESC_cluster_r_state | One required, as follows: r_name Value type: string Optional: none | Registers for all state change events for resource name |
EC_Cluster None | Required: none Optional: none | Registers for all Sun Cluster events |