The 512-byte page size had been selected for the System/38 because the main memory sizes were constrained.4 But a 512-byte page was too small for the AS/400’s larger main-memory sizes. There were too many pages, and our page tables were getting too large. Besides, for years we have been blocking these small pages into larger “logical” pages to reduce disk activity. For the new processors, we decided to increase the size of the pages to 4K.
4An IBM “expert” came to Rochester before the System/38 was announced and declared that our planned memory sizes were too large. Over the protests of those of us who knew better, we were forced to reduce our memory sizes. The original Model 5 was a poor performer because it was constrained to 2 MB of memory. When memory sizes increased, so did performance. When the same expert came back to help us with the AS/400, we threw him out.
We retained the 520-byte sector size on disk for the new models. A 4K page is now stored on eight consecutive sectors. With eight 8-byte headers for each page, there is more than enough room to store the 256 tag bits that a 4K page can contain.
We also added a special tag register to the PowerPC processor architecture to be used for extracting tags from memory. When the processor executes another of the special tag instructions, called Load Multiple Doubleword (lmd), up to 16 doublewords (eight quadwords) from memory can be read into 16 registers. The eight tag bits from the memory quadwords are stored in the tag register as part of this instruction. Note that the logical eight bits are stored in the tag register, not the 16 physical tag bits that would be found in memory. Recall that a quadword is two 64-bit (8-byte) memory words; therefore, there are two physical tag bits in a quadword, which is how we get 16 physical tag bits in eight quadwords. The tag bits for the page can be collected using this instruction, so they can be written to disk with the data. Again, this instruction is available only in the tags-active mode.
Inside a Pointer
We use a pointer to access objects on the AS/400. In this section, we concentrate only on the format of a resolved pointer, which provides two functions. A resolved pointer describes the object and the authority the user has to the object; it also provides the address of the object in the single-level store. If we look inside one of these 16-byte pointers, we find it divided into two 8-byte parts. The first part contains the description of the object, and the second part contains the virtual address.
The first part of a resolved pointer contains status bits that, among other things, identify whether this is a system, space, data, instruction, or procedure pointer. There is also information in this first part about the object or data accessed by the pointer. For example, a system pointer identifies the MI system object type and the OS/400-defined subtype for the object. A data pointer describes the type of data to which it points. The final piece of information contained in the first part of the pointer is the authority to the object. As previously noted, the authority field is used only when the pointer is owned by the operating system in the system state. Pointers created by user programs in the user state do not have the authority field filled in.
The information in the first part of the pointer (just described) does not occupy all 8 bytes; it occupies only 4 bytes. This means there are 4 bytes in the pointer that are unused. We have reserved this space for future expansion of the address. This extra 32 bits, together with the 64 address bits in the pointer, gives us the opportunity to expand the AS/400 address to 96 bits (12 bytes) without affecting any program above the MI. If we wanted more space, we could even move the type and authority information out of the pointer and go beyond 96 bits.
The second part of a resolved pointer contains a 64-bit address. What’s more, it has always contained a 64-bit address, and this has caused confusion for years over the size of the virtual address in a System/38 and an AS/400. Is it 48 bits or 64 bits? Today, with the new RISC models of the AS/400, the answer is simple: It is a 64-bit address. For the System/38 and earlier models of the AS/400, it was both. The MI always had a 64-bit address, and the hardware always had a 48-bit address. How these worked together is our next topic.
A Tale of Two Address Sizes
The introduction of the new AS/400 RISC models with their full 64-bit hardware addresses has eliminated most of the problems and a lot of the confusion that existed with the mixed 64-bit and 48-bit addressing structure of the previous models. To appreciate why the 64-bit hardware address is so significant, we need to look at the original 48-bit implementation.
The 48-bit address came about as a compromise. The System/38 operating system designers decided they wanted a 64-bit address. Once the size of a pointer had been set to 16 bytes, there was plenty of room for the 64-bit address. At the hardware level, we had a different problem. The more bits we had in the address, the larger the size of the registers in the processor. Larger registers meant more circuits and more hardware costs.
Ray Klotz was our engineering manager for the System/38. Ray wasn’t convinced we needed such a large address. We had many conversations on this topic, but he wasn’t buying the arguments for a persistent, 64-bit address that would last forever. In the mid-1970s, the IBM mainframe division was about to announce a new, larger address for the System/370, called extended addressing (XA). The System/370 address was about to grow from 24 to 31 bits with XA. Ray accused us of trying to build a system architecture that was larger than the System/370 — he argued that 32 bits was bigger than 31. We could beat them by 1 bit. “You don’t need to beat them by 33 bits,” he would say.
We argued a 32-bit address would not work for the single-level store of the System/38. We needed 64 bits. When it became obvious we were making no progress in reaching a decision, we did what anyone who has ever haggled over the price of a used car has done. We split the difference between 32 and 64 bits. From that point on we would have a 48-bit address.
To support the segmented memory, the 48-bit address is divided in two. The high-order bits of the address identify the segment. This high part of the address is called the segment identifier (SID). The low-order bits identify the byte in the segment, which is called the offset. In the hardware, we decided to use the high-order 32 bits for the SID and the low-order 16 bits for the offset. For notation purposes, we will call this the 32/16 view of the address. The 16 bits in the offset means a segment size of 64K (216 = 64K).