Data Service API and Data Service Development Library API
The SunPlex system provides the following to make applications highly available:
Data services supplied as part of the SunPlex system
A data service API
A data service development library API
A "generic" data service
The Sun Cluster 3.1 Data Services Installation and Configuration Guide describes how to install and configure the data services supplied with the SunPlex system. The Sun Cluster 3.1 Data Services Developer's Guide describes how to instrument other applications to be highly available under the Sun Cluster framework.
The Sun Cluster APIs enable application programmers to develop fault monitors and scripts that start and stop data services instances. With these tools, an application can be instrumented to be a failover or a scalable data service. In addition, the SunPlex system provides a "generic" data service that can be used to quickly generate an application's required start and stop methods to make it run as a failover or scalable service.
Using the Cluster Interconnect for Data Service Traffic
A cluster must have multiple network connections between nodes, forming the cluster interconnect. The clustering software uses multiple interconnects both for high availability and to improve performance. For internal traffic (for example, file system data or scalable services data), messages are striped across all available interconnects in a round-robin fashion.
The cluster interconnect is also available to applications, for highly available communication between nodes. For example, a distributed application might have components running on different nodes that need to communicate. By using the cluster interconnect rather than the public transport, these connections can withstand the failure of an individual link.
To use the cluster interconnect for communication between nodes, an application must use the private hostnames configured when the cluster was installed. For example, if the private hostname for node 1 is clusternode1-priv, use that name to communicate over the cluster interconnect to node 1. TCP sockets opened using this name are routed over the cluster interconnect and can be transparently re-routed in the event of network failure.
Note that because the private hostnames can be configured during installation, the cluster interconnect can use any name chosen at that time. The actual name can be obtained from scha_cluster_get(3HA) with the scha_privatelink_hostname_node argument.
For application-level use of the cluster interconnect, a single interconnect is used between each pair of nodes, but separate interconnects are used for different node pairs, if possible. For example, consider an application running on three nodes and communicating over the cluster interconnect. Communication between nodes 1 and 2 might take place on interface hme0, while communication between nodes 1 and 3 might take place on interface qfe1. That is, application communication between any two nodes is limited to a single interconnect, while internal clustering communication is striped over all interconnects.
Note that the application shares the interconnect with internal clustering traffic, so the bandwidth available to the application depends on the bandwidth used for other clustering traffic. In the event of a failure, internal traffic can round-robin over the remaining interconnects, while application connections on a failed interconnect can switch to a working interconnect.
Two types of addresses support the cluster interconnect, and gethostbyname(3N) on a private hostname normally returns two IP addresses. The first address is called the logical pairwise address, and the second address is called the logical pernode address.
A separate logical pairwise address is assigned to each pair of nodes. This small logical network supports failover of connections. Each node is also assigned a fixed pernode address. That is, the logical pairwise addresses for clusternode1-priv are different on each node, while the logical pernode address for clusternode1-priv is the same on each node. A node does not have a pairwise address to itself, however, so gethostbyname(clusternode1-priv) on node 1 returns only the logical pernode address.
Note that applications accepting connections over the cluster interconnect and then verifying the IP address for security reasons must check against all IP addresses returned from gethostbyname, not just the first IP address.
If you need consistent IP addresses in your application at all points, configure the application to bind to the pernode address on both the client and the server side so that all connections can appear to come and go from the pernode address.
Resources, Resource Groups, and Resource Types
Data services utilize several types of resources: applications such as Apache Web Server or Sun ONE Web Server utilize network addresses (logical hostnames and shared addresses) upon which the applications depend. Application and network resources form a basic unit that is managed by the RGM.
Data services are resource types. For example, Sun Cluster HA for Oracle is the resource type SUNW.oracle-server and Sun Cluster HA for Apache is the resource type SUNW.apache.
A resource is an instantiation of a resource type that is defined cluster wide. There are several resource types defined.
Network resources are either SUNW.LogicalHostname or SUNW.SharedAddress resource types. These two resource types are pre-registered by the Sun Cluster software.
The SUNW.HAStorage and HAStoragePlus resource types are used to synchronize the startup of resources and disk device groups upon which the resources depend. It ensures that before a data service starts, the paths to cluster file system mount points, global devices, and device group names are available. For more information, see "Synchronizing the Startups Between Resource Groups and Disk Device Groups" in the Data Services Installation and Configuration Guide. (The HAStoragePlus resource type became available in Sun Cluster 3.0 5/02 and added another feature, enabling local file systems to be highly available. For more information on this feature, see HAStoragePlus Resource Type.)
RGM-managed resources are placed into groups, called resource groups, so that they can be managed as a unit. A resource group is migrated as a unit if a failover or switchover is initiated on the resource group.
Note - When you bring a resource group containing application resources online, the application is started. The data service start method waits until the application is up and running before exiting successfully. The determination of when the application is up and running is accomplished the same way the data service fault monitor determines that a data service is serving clients. Refer to the Sun Cluster 3.1 Data Services Installation and Configuration Guide for more information on this process.
Resource Group Manager (RGM)
The RGM controls data services (applications) as resources, which are managed by resource type implementations. These implementations are either supplied by Sun or created by a developer with a generic data service template, the Data Service Development Library API (DSDL API), or the Resource Management API (RMAPI). The cluster administrator creates and manages resources in containers called resource groups. The RGM stops and starts resource groups on selected nodes in response to cluster membership changes.
The RGM acts on resources and resource groups. RGM actions cause resources and resource groups to move between online and offline states. A complete description of the states and settings that can be applied to resources and resource groups is in the section Resource and Resource Group States and Settings. Refer to Resources, Resource Groups, and Resource Types for information about how to launch a resource management project under RGM control.
Resource and Resource Group States and Settings
An administrator applies static settings to resources and resource groups. These settings can only be changed through administrative actions. The RGM moves resource groups between dynamic "states." These settings and states are described in the following list.
Managed or unmanaged - These are cluster-wide settings that apply only to resource groups. Resource groups are managed by the RGM. The scrgadm(1M) command can be used to cause the RGM to manage or to unmanage a resource group. These settings do not change with a cluster reconfiguration.
When a resource group is first created, it is unmanaged. It must be managed before any resources placed in the group can become active.
In some data services, for example a scalable web server, work must be done prior to starting up network resources and after they are stopped. This work is done by initialization (INIT) and finish (FINI) data service methods. The INIT methods only run if the resource group in which the resources reside is in the managed state.
When a resource group is moved from unmanaged to managed, any registered INIT methods for the group are run on the resources in the group.
When a resource group is moved from managed to unmanaged, any registered FINI methods are called to perform cleanup.
The most common use of INIT and FINI methods are for network resources for scalable services, but they can be used for any initialization or cleanup work that is not done by the application.
Enabled or disabled - These are cluster-wide settings that apply to resources. The scrgadm(1M) command can be used to enable or disable a resource. These settings do not change with a cluster reconfiguration.
The normal setting for a resource is that it is enabled and actively running in the system.
If for some reason, you want to make the resource unavailable on all cluster nodes, you disable the resource. A disabled resource is not available for general use.
Online or offline - These are dynamic states that apply to both resource and resource groups.
These states change as the cluster transitions through cluster reconfiguration steps during switchover or failover. They can also be changed through administrative actions. The scswitch(1M) can be used to change the online or offline state of a resource or resource group.
A failover resource or resource group can only be online on one node at any time. A scalable resource or resource group can be online on some nodes and offline on others. During a switchover or failover, resource groups and the resources within them are taken offline on one node and then brought online on another node.
If a resource group is offline then all of its resources are offline. If a resource group is online, then all of its enabled resources are online.
Resource groups can contain several resources, with dependencies between resources. These dependencies require that the resources be brought online and offline in a particular order. The methods used to bring resources online and offline might take different amounts of time for each resource. Because of resource dependencies and start and stop time differences, resources within a single resource group can have different online and offline states during a cluster reconfiguration.
Resource and Resource Group Properties
You can configure property values for resources and resource groups for your SunPlex data services. Standard properties are common to all data services. Extension properties are specific to each data service. Some standard and extension properties are configured with default settings so that you do not have to modify them. Others need to be set as part of the process of creating and configuring resources. The documentation for each data service specifies which resource properties can be set and how to set them.
The standard properties are used to configure resource and resource group properties that are usually independent of any particular data service. The set of standard properties is described in an appendix to the Sun Cluster 3.1 Data Services Installation and Configuration Guide.
The RGM extension properties provide information such as the location of application binaries and configuration files. You modify extension properties as you configure your data services. The set of extension properties is described in the individual chapter for the data service in the Sun Cluster 3.1 Data Services Installation and Configuration Guide.