Running Out of Addresses
Because we had decided not to reuse the 48-bit address except at IPL boundaries in the System/38, there was a question of whether we could run out of addresses. We did some original calculations to see if we had a problem. According to these figures, with one IPL per day, 365 days per year, it would be 180 years before we reused the IPL number. Running out of SID extenders was not a problem. By knowing the projected performance of our future processors, we could calculate the maximum number of 64K segments we could generate between daily IPLs. These calculations also showed there would be no problem. But all these calculations were wrong, and some large AS/400 systems started to run out of addresses.5
5As the one who made those original calculations and reached the conclusion that there would not be a problem, I have taken a lot of heat in recent years. I have also been highly motivated to move the AS/400 to a 64-bit hardware address.
What happened? First, one IPL per day may have been a good assumption for the early System/38, but as businesses began to need their computers 24 hours a day, there was no time for IPLs. Also, the size of our systems had grown to the point that the IPL time was too long. We have since made major changes to the AS/400 to reduce IPL time to just a few minutes. But even with the changes, one IPL per day is still an unreasonable assumption. The second problem with the calculation was the assumption of a 64K segment. With the 24/24 view of the address, the original VMC always created 16 MB segments.
To further aggravate the problem, the main storage management component of the original VMC had divided the entire 48-bit virtual address space into quadrants. The top two bits of the 48-bit address were used to identify the quadrant. Addresses were assigned from one of these quadrants based on their intended use. One quadrant was reserved for addresses of permanent objects, so all permanent addresses had the same setting in the first two bits. Another quadrant was reserved for temporary object addresses. A third was reserved for addresses of temporary objects in an access group. In Chapter 5, we introduced an access group, which is another system object that allows multiple temporary objects of different types to be bundled and accessed together. Readers may be familiar with a process access group (PAG). Because permanent objects cannot be in an access group, the fourth quadrant was not used. This original decision was based on the assumption that a 48-bit address space was so big, one quarter of it could be thrown away with no loss. Of the 256 TB (248 bytes) of virtual memory we liked to brag about, 64 TB were never used in the System/38 and the early AS/400 models.
The problem that occurred with some large AS/400 systems was that they were running out of temporary addresses. This was called SID wrap, because all the SIDs available during an IPL were used up. With the address space divided into quadrants, there were only 4 million 16 MB temporary segments per IPL. Consequently, large systems with applications that used lots of temporary objects could run out of temporary SIDs if they were not IPLed for several days. The fix from a customer viewpoint was to re-IPL the system. This solved the immediate problem but was not a long-term solution.
During the first few operating-system releases of the AS/400, changes were made to the system software to reduce the SID wrap problem. Components began to use 64K segments, whenever possible, instead of 16 MB segments. The quarter of the address space that had been thrown away was also reclaimed. These system software changes did not fix the problem, however; they only gave us some breathing room until the 64-bit RISC processors arrived.
We have not discussed permanent addresses. Their maximum number is limited by the amount of disk space attached to the system, because we always keep some physical disk storage even for deleted objects. We limited the total amount of disk space that could be attached to an AS/400, so that a customer would not run out of permanent addresses. Running out of permanent addresses cannot be fixed with an IPL. The solution requires the entire system to be reloaded, and that is totally unacceptable for anyone. We therefore limited the disk sizes in previous non-RISC AS/400 models.
We’ve spent the past several paragraphs describing the addressing structure of the System/38 and the early models of the AS/400 to show the real reason why IBM chose to move to the RISC processors. We knew even before the introduction of the original AS/400 that the 48-bit address would limit the future growth of the system. As systems got larger and faster, they were going to use more temporary addresses. Customers also wanted to attach more disks to these systems.
There was no way the 48-bit address was going take us far into the future, though it did take some time to convince management this was the case. Rochester has always followed the old adage, “If it ain’t broke, don’t fix it”; but we finally convinced them it was broken. The good news was that the breakage was inside the system. The AS/400’s technology independence protected our customers.
When you break the address in a computer architecture, you might as well change everything. We now had the chance to get rid of the IMPI and move to a RISC design. Our original RISC processor that we started to design in 1990, which we called C-RISC (the C was for commercial), had a 96-bit address. We had the room for this large address in the pointers and decided to go all the way. When we decided to use the PowerPC architecture in 1991, we scaled the address back to 64 bits.
Address Translation
The virtual address in a computer system must be translated into a real address before the memory can be accessed. Earlier, we saw that the Power-PC architecture has another level of address, called the effective address, which programs generate. These effective addresses must first be translated into virtual addresses before the virtual addresses can be translated to real addresses. In this section, we discuss how these translations are accomplished. Some of the details may get a little spicy.