Campus Clustering with Sun Cluster Software -- Concepts
This appendix introduces some of the basic concepts of campus clustering and provides some configuration and setup examples. This appendix does not attempt to explain clustering, provide information on clustering administration, or furnish details about hardware installation and configuration. For conceptual information on clustering, see your Sun Cluster concepts documentation and your Sun Cluster system administration documentation.
Introduction
The only significant difference between traditional clustering and campus clustering is that of distance. In campus clustering, the nodes of a cluster configuration can be up to several kilometers apart. This increases the likelihood that, in the case of a catastrophe such as a fire or earthquake, at least one server and its storage will survive.
Sun Cluster software supports up to three nodes in a campus cluster configuration, and both two and three-room configuration are supported. A room can be thought of as a functionally independent hardware grouping (such as a node and its attendant storage, or a quorum device physically separated from any nodes) that has been separated from other rooms to increase the likelihood of failover and redundancy in case of accident or failure. The definition of a room therefore depends on the type of failures to be safeguarded against, as indicated by Table A-1.
Table A-1 Definitions of "Room"
Failure Scenario | Sample Definitions of Separate Rooms |
|---|---|
Power line failure | Isolated and independent power supplies |
Minor accidents, furniture collapse, seepage, etc. | Different parts of physical room |
Small fire. Sprinklers (fire) starting | Different physical areas (for example, sprinkler zone) |
Structural failure (building-wide fire, for example) | Different buildings |
Large-scale natural disaster (for example, earthquake or flood) | Different corporate campuses up to several kilometers apart |
In two-room configurations, the quorum disk occupies the same room as one node (see Example Two-Room Configuration With Host-Based Data Replication). In the case of a three-room configuration, the third room is used for the quorum disk (Example Three-Room Configuration) or a small server.
In a two-room configuration, the quorum disk should be located in the room that is more likely to survive an accident or catastrophe in the event that all cluster transport and disk connectivity is lost between rooms. (If only cluster transport is lost, the node sharing a room with the quorum disk will not necessarily be the room that reserves the quorum disk first.) For more information about quorum and quorum devices, see the Sun Cluster concepts documentation.
The advantage to a three-room cluster is that, if any one of the three rooms is lost, automatic failover should typically be possible; whereas with a two-room cluster, if an entire room is lost, automatic failover is possible only if the surviving room contains the quorum disk. Only a three-room configuration guarantees system availability in the event of complete loss of an entire room (in the absence of any other failures).
Note - As with noncampus configurations, data integrity will be compromised if there are other unrecovered I/O failures present when a room is destroyed, and the most up-to-date submirror was in the destroyed room.
In a campus cluster configuration, each of the two rooms used for nodes should have an equal number of shared disks. (In a two-room configuration, one room can have a separate quorum disk, so the two rooms need not have the same number of total disks.) Data replication between shared disks must always be performed across rooms, rather than within rooms. In other words, both copies of the data should never be in the same room. Mirroring is required for all campus cluster configurations, since RAID-5 alone does not lend itself to providing data redundancy across rooms.
If you use Solstice DiskSuite/Solaris Volume Manager as your volume manager for shared device groups, pay special consideration to the distribution of replicas. In two-room configurations, all disksets should be configured with an additional replica in the room that houses the cluster quorum disk. Further, all Solstice DiskSuite/Solaris Volume Manager device groups should be configured to use the node in the quorum disk room as their default primary room. In three-room configurations, the third room should not only house the quorum disk, but also include at least one extra disk configured into each of the disksets. Each diskset should include a third-room disk with an extra Solstice DiskSuite/Solaris Volume Manager replica per diskset. Quorum disks can be used as metadb replicas in a metaset.
Sun Cluster software supports campus cluster configurations with rooms up to 10km apart by using FC over single-mode fiber. Sun Cluster supports campus cluster configurations separated by greater distances by using Nortel OPTera wavelength division multiplexing (WDM) products. For more information regarding Nortel OPTera WDM products, see the Nortel OPTera product documentation.
Campus Cluster Configuration Examples
This section provides examples of two and three-room campus clustering configurations:
Figure A-1 shows a two-room configuration that is using Sun StorEdge T3 partner groups for host-based mirroring.
Figure A-2 shows a three-room campus cluster that is using Sun StorEdge A5x00 disk arrays for host-based mirroring.
Note - The examples in this chapter illustrate general configurations and are not intended to indicate required or recommended setups. The type of storage array shown, for example, is purely for illustrative purposes. For simplicity's sake, the diagrams and explanations concentrate only on features unique to understanding campus clustering, so that, for example, public-network Ethernet connections are not shown.
See Additional Campus Cluster Configuration Examples for other example setups.
Example Two-Room Configuration With Host-Based Data Replication
A two-room configuration with host-based data replication is defined as follows.
Two separate rooms
Both rooms with one node each and disk subsystems
Data mirrored across disk subsystems in these rooms
At least one disk subsystem, attached to both hosts, used as a quorum device, located in one of the rooms
In the event of loss of the room containing the quorum disk, the system will be unable to recover automatically. Recovery will require operator intervention.
Figure A-1 shows a sample two-room configuration by using a partner-group of Sun StorEdge T3/T3+ disk arrays in each room.
Figure A-1 Example Two-Room Campus Cluster without a multipathing solution implemented
Figure A-1 is similar to a standard noncampus configuration; the most obvious difference is that FC switches have been added to switch from multimode to single-mode fibers.
Although not so depicted in this chapter, campus clustering enables for configurations by using multiple storage arrays. Where large storage environments are required, additional SAN switches for Sun StorEdge T3/T3+ arrays might be required.
Example Three-Room Configuration
Sun Cluster supports two different three-room configurations.
Two rooms with one node each and an equal number of disk arrays, in this case, Sun StorEdge A5x00 disk subsystems; data is mirrored across disk-subsystems in these rooms
Third room with at least one disk subsystem attached to both hosts to be used as a quorum device
Two rooms contain one node each and an equal number of disk arrays. The third room contains a small server. The small server removes the need for a storage array to be used as a quorum device.
Figure A-2 shows this configuration by using Sun StorEdge A5x00 disk arrays. Note that, unlike Figure A-1, which used Sun StorEdge T3 disk trays, this configuration does not use FC switches to connect Sun StorEdge A5x00 disk arrays. (Switches are not needed, as long-wave GBICs are already present in the A5x00s and should also be present in the servers' host bus adapters.)
Figure A-2 Example Three-Room Configuration
In this configuration, as long as at least two rooms are up and communicating, recovery will be automatic. This is the only configuration in which loss of any one room is guaranteed to be automatically handled. Loss of two rooms requires the replacement or rebuilding of one room and typically requires Sun Service intervention.
Campus Cluster Interconnect Hardware
The following components make up the campus cluster interconnect hardware configuration:
SunFastEthernet Adapter. The SunFastEthernet Adapter provides 10/100 Mbps Ethernet functionality with an RJ-45 connector.
Media converters. Use RJ-45 media converters to convert from copper to optical fiber connectors. Consult your Sun sales representative for additional information about media converters.
Two 9/125-micrometer single-mode fiber pairs.
Figure A-3 shows the setup for the cluster interconnect:
Figure A-3 100BASE-FX Setup
