Key Concepts - Administration and Application Development
This chapter describes the key concepts related to the software components of a SunPlex system. The topics covered include:
Cluster Administration and Application Development
This information is directed primarily toward system administrators and application developers using the SunPlex API and SDK. Cluster system administrators can use this information as background to installing, configuring, and administering cluster software. Application developers can use the information to understand the cluster environment in which they will be working.
The following figure shows a high-level view of how the cluster administration concepts map to the cluster architecture.
Figure 3-1 Sun Cluster Software Architecture
Administrative Interfaces
You can choose how you install, configure, and administer the SunPlex system from several user interfaces. You can accomplish system administration tasks either through the SunPlex Manager graphic user interface (GUI), or through the documented command-line interface. On top of the command-line interface are some utilities, such as scinstall and scsetup, to simplify selected installation and configuration tasks. The SunPlex system also has a module that runs as part of Sun Management Center that provides a GUI to certain cluster tasks. Refer to the introductory chapter in the Sun Cluster 3.1 System Administration Guide for complete descriptions of the administrative interfaces.
Cluster Time
Time between all nodes in a cluster must be synchronized. Whether you synchronize the cluster nodes with any outside time source is not important to cluster operation. The SunPlex system employs the Network Time Protocol (NTP) to synchronize the clocks between nodes.
In general, a change in the system clock of a fraction of a second causes no problems. However, if you run date(1), rdate(1M), or xntpdate(1M) (interactively, or within cron scripts) on an active cluster, you can force a time change much larger than a fraction of a second to synchronize the system clock to the time source. This forced change might cause problems with file modification timestamps or confuse the NTP service.
When you install the Solaris operating environment on each cluster node, you have an opportunity to change the default time and date setting for the node. In general, you can accept the factory default.
When you install Sun Cluster software using scinstall(1M), one step in the process is to configure NTP for the cluster. Sun Cluster software supplies a template file, ntp.cluster (see /etc/inet/ntp.cluster on an installed cluster node), that establishes a peer relationship between all cluster nodes, with one node being the "preferred" node. Nodes are identified by their private host names and time synchronization occurs across the cluster interconnect. The instructions for how to configure the cluster for NTP are included in the Sun Cluster 3.1 Software Installation Guide.
Alternately, you can set up one or more NTP servers outside the cluster and change the ntp.conf file to reflect that configuration.
In normal operation, you should never need to adjust the time on the cluster. However, if the time was set incorrectly when you installed the Solaris operating environment and you want to change it, the procedure for doing so is included in the Sun Cluster 3.1 System Administration Guide.
High-Availability Framework
The SunPlex system makes all components on the "path" between users and data highly available, including network interfaces, the applications themselves, the file system, and the multihost disks. In general, a cluster component is highly available if it survives any single (software or hardware) failure in the system.
The following table shows the kinds of SunPlex component failures (both hardware and software) and the kinds of recovery built into the high-availability framework.
Table 3-1 Levels of SunPlex Failure Detection and Recovery
Failed Cluster Component | Software Recovery | Hardware Recovery |
|---|---|---|
Data service | HA API, HA framework | N/A |
Public network adapter | IP Network Multipathing | Multiple public network adapter cards |
Cluster file system | Primary and secondary replicas | Multihost disks |
Mirrored multihost disk | Volume management (Solaris Volume Manager and VERITAS Volume Manager) | Hardware RAID-5 (for example, Sun StorEdge™ A3x00) |
Global device | Primary and secondary replicas | Multiple paths to the device, cluster transport junctions |
Private network | HA transport software | Multiple private hardware-independent networks |
Node | CMM, failfast driver | Multiple nodes |
Sun Cluster software's high-availability framework detects a node failure quickly and creates a new equivalent server for the framework resources on a remaining node in the cluster. At no time are all framework resources unavailable. Framework resources unaffected by a crashed node are fully available during recovery. Furthermore, framework resources of the failed node become available as soon as they are recovered. A recovered framework resource does not have to wait for all other framework resources to complete their recovery.
Most highly available framework resources are recovered transparently to the applications (data services) using the resource. The semantics of framework resource access are fully preserved across node failure. The applications simply cannot tell that the framework resource server has been moved to another node. Failure of a single node is completely transparent to programs on remaining nodes using the files, devices, and disk volumes attached to this node, as long as an alternative hardware path exists to the disks from another node. An example is the use of multihost disks that have ports to multiple nodes.
Cluster Membership Monitor
The Cluster Membership Monitor (CMM) is a distributed set of agents, one per cluster member. The agents exchange messages over the cluster interconnect to:
Enforce a consistent membership view on all nodes (quorum)
Drive synchronized reconfiguration in response to membership changes, using registered callbacks
Handle cluster partitioning (split brain, amnesia)
Ensure full connectivity among all cluster members
Unlike previous Sun Cluster software releases, CMM runs entirely in the kernel.