Chapter 3

Installing and Maintaining Cluster Interconnect Hardware

This chapter describes the procedures to install and maintain cluster interconnect hardware. Where appropriate, this chapter includes separate procedures for the three supported varieties of Sun Cluster interconnect: Ethernet, peripheral component interconnect-scalable coherent interface (PCI-SCI), and Sun Fire Link.

This chapter contains procedures and guidelines on how to maintain cluster interconnect hardware. This chapter contains the following information.

For conceptual information about cluster interconnects, see your Sun Cluster concepts documentation.

For information about how to administer cluster interconnects, see your Sun Cluster system administration documentation.

Installing Ethernet Cluster Interconnect Hardware

Table 3-1 lists procedures about how to install Ethernet cluster interconnect hardware. Perform the procedures in the order that the procedures are listed. This section contains a procedure about how to install cluster hardware during an initial installation of a cluster, before you install Sun Cluster software.

Table 3-1 Task Map: Installing Ethernet Cluster Interconnect Hardware

Task	For Instructions, Go To
Install the transport adapters.	The documentation that shipped with your nodes and host adapters
Install the cluster transport cables. For clusters with more than two nodes, also install the transport junctions.	How to Install Ethernet Transport Cables and Transport Junctions

How to Install Ethernet Transport Cables and Transport Junctions

Use this procedure to install Ethernet transport cables and transport junctions (switches).

If not already installed, install transport adapters in your cluster nodes.
For the procedure about how to install transport adapters, see the documentation that shipped with your host adapters and node hardware.

Install the transport cables. If necessary, install transport junctions.
- As Figure 3-1 shows, a cluster with only two nodes can use a point-to-point connection, requiring no transport junctions. Use a point-to-point (crossover) Ethernet cable if you are connecting 100BaseT or TPE ports of a node directly to ports on another node. Gigabit Ethernet uses the standard fiber-optic cable for both point-to-point and switch configurations.
  
  Figure 3-1 Typical Two-Node Cluster Interconnect
  
  Note - If you use a transport junction in a two-node cluster, you can add additional nodes to the cluster. You can add additional nodes without bringing the cluster offline to reconfigure the transport path.
- As Figure 3-2 shows, a cluster with more than two nodes require two transport junctions. These transport junctions are Ethernet switches (customer-supplied).
  
  Figure 3-2 Typical Four-Node Cluster Interconnect

Where to Go From Here

Install the cluster software. Configure the interconnect after you install all other hardware. To review the task map about how to install cluster hardware and software, see Table 3-1.

Installing PCI-SCI Cluster Interconnect Hardware

Table 3-2 lists procedures about how to install PCI-SCI cluster interconnect hardware. Perform the procedures in the order that the procedures are listed. This section contains a procedure about how to install cluster hardware during an initial installation of a cluster before you install Sun Cluster software.

Table 3-2 Task Map: Installing PCI-SCI Cluster Interconnect Hardware

Task	For Instructions, Go To
Install the PCI-SCI transport cables. For four-node clusters, also install a PCI-SCI transport junction.	How to Install PCI-SCI Transport Cables and Transport Junction

How to Install PCI-SCI Transport Cables and Transport Junction

Use this procedure to install PCI-SCI transport cables and transport junctions (switches).

When you perform this procedure, the following error messages display on your console.

Solaris 8

Nov 13 20:11:43 e04a ip: ip_rput_dlpi(scid0): DL_ERROR_ACK for
DL_ENABMULTI_REQ(29), errno 7, unix 0
Nov 13 20:11:43 e04a ip: ip: joining multicasts failed (7) on scid0
 - will use link layer broadcasts for multicast

These error messages display because the SCI dlpi driver does not support the multicast feature. These error messages display when the ip module probes the driver. Sun Cluster software does not use the multicast feature on the private interconnect. You can safely ignore these error messages.

Solaris 9

Dec  4 17:40:14 e03a in.routed[132]: write(rt_sock) RTM_ADD 172.17.0.128/25
-->172.17.0.129    metric=0 flags=0: File exists
Dec  4 17:40:19 e03a in.routed[132]: interface scid0 to 172.17.0.129 broken:
in=0 ierr=0 out=0 oerr=4

These error messages are responses to the way Solaris 9 handles SCI dlpi interfaces. Solaris 9 uses the in.routed routing protocol as the default routing protocol. You can safely ignore these error messages. The in.routed routing protocol is the source of these error messages.

If not already installed, install PCI-SCI transport adapters in your cluster nodes.
For the procedure about how to install PCI-SCI transport adapters and set their DIP switches, see the documentation that shipped with your PCI-SCI host adapters and node hardware.

Note - Sbus-SCI host adapters are not supported by Sun Cluster software. If you are upgrading from a Sun Cluster 2.2 cluster, remove any Sbus-SCI host adapters from the cluster nodes. If you do not remove these adapters, you might see panic error messages during the SCI self test.

Install the PCI-SCI transport cables and optionally, transport junctions, depending about how many nodes are in your cluster.
- A two-node cluster can use a point-to-point connection, requiring no transport junction. See Figure 3-3.
  
  Connect the ends of the cables marked SCI Out to the O connectors on the adapters.
  
  Connect the ends of the cables marked SCI In to the I connectors of the adapters as shown in Figure 3-3.
  
  Figure 3-3 Typical Two-Node PCI-SCI Cluster Interconnect
- A four-node cluster requires SCI transport junctions. See Figure 3-4 for a cabling diagram. For the procedure about how to install and cable, see the SCI switch documentation that shipped with your hardware the switches.
  
  Connect the ends of the cables that are marked SCI Out to the O connectors on the adapters and the Out connectors on the transport junctions.
  
  Connect the ends of the cables that are marked SCI In to the I connectors of the adapters and In connectors on the transport junctions, as shown in Figure 3-4.
  
  Note - Set the Unit selectors on the fronts of the SCI transport junctions to F. Do not use the X-Ports on the SCI transport junctions.
  
  Figure 3-4 Typical Four-Node PCI-SCI Cluster Interconnect