Storage Management Related Concepts

This chapter examines the basic elements involved in storage subsystems, and explains basic concepts related to the hardware and software involved.

Overview

Storage subsystems may contain a variety of hardware and software products, but primarily exist to supplement the relatively expensive main memory of a computer with less expensive non-volatile storage. Secondary functions include information exchange via removable storage media, backup of vital information for recovery in the event of failure, and short term storage of frequently accessed information. This overview will explain the rationale and some of the concepts involved in the development and usage of the elements of storage subsystems.

General Concepts

A computer system is composed of a number of different subsystems that cooperate to carry out tasks on behalf of a user. Generally speaking, the process works as follows. A user wishes to run an application to take some input, operate on it, and produce some output. The application exists as a series of instructions to the Central Processing Unit (CPU) of the computer system that tell it where to get information, what to do with it, and where to put the results. For the CPU to load these instructions and execute them, they must be located in the main memory of the computer system. The first problem is that main memory is volatile, which means that without power, it will lose whatever was stored in it. Thus when the computer is first powered on, it must load its instructions from a non-volatile source, and this is the first function of the storage subsystem, to provide the CPU with it's operating system, and then access to applications and data for processing. This function is usually provided by magnetic disks which are relatively fast, and allow direct access to required information, although a brand new system may load it's instructions initially from diskette, or more commonly magnetic tape.

Now the CPU has access to the required instructions and data, and can go ahead and do useful work for a user. So much so however, that pretty soon the user wants more work done, in parallel, and to support other users workloads. Clever operating system design allows this multitasking, enabling many things to be done at once, but the next problem is that there is not enough volatile main memory to contain all the different applications and data that are in concurrent use. One solution would be to keep purchasing more memory to enable all required applications and data to be maintained in main memory, but this quickly becomes prohibitively expensive. A better solution is to use a system called paging which utilizes reasonably fast, much cheaper magnetic disk as an extension to the main memory. An area of the disk is set aside as paging space, and those applications and data, or parts of applications and data, that are not currently in use, are stored in this space, or swapped out. When an instruction calls for a piece of this data, or the next instruction exists in the page space, the page containing the required information is swapped in, or copied from the disk back into main memory, and something else temporarily not required is copied out. Page space is so named because the main memory is divided into sections known as pages, and these form the units of exchange.

Soon there are large amounts of vital data being produced by the computer, which immediately develops a fault causing everything to be lost, and creating a great many angry people. Not so. The third function of a storage subsystem is to provide inexpensive non-volatile storage for copies of vital information that can be restored to the system in the event of failure. Removable media, such as magnetic tape, diskette, or optical disks are generally used for this, as transfer to and from the devices is easy (though much slower than disk or main memory), and the media can then be stored safely in a secure place for use in the event of failure.

With the advent of faster and faster low cost, high capacity devices such as tape and optical, a fourth usage of storage subsystems is emerging. Space management is the monitoring of information usage, so that less frequently accessed information can be moved to lower cost, slower storage, thereby maximizing the use of the higher cost, faster devices.

One final use of storage devices is for information exchange. Applications, data, and even large directories of information can be stored on removable media, such as optical, tape, diskette, or disk pack. The media can then be removed and replaced for access on a different system, thereby allowing simple transfer for common access to information. With the increase in performance and reduction in the cost of communications networks, this function is becoming less necessary.


Figure: Storage Subsystem Component Usage

Hardware Concepts

As has been described in the previous section, storage subsystems components have evolved to meet specific requirements within the computer system, and each have different characteristics that enable these requirements to be met. A storage subsystem may contain any or all of the following types of device.

  1. Diskette Storage

    Diskettes, or floppy disks, were among the first permanent storage devices developed. Information can be written to and read from a diskette via a diskette drive unit attached to the computer system. The diskette can then be removed and reinserted into a drive on another computer, where the information can be utilized. Information can be accessed directly from anywhere on the diskette so individual files can be quickly located and accessed if required.

    There are quite a few different standards for diskettes, both in terms of physical specifications, and information capacities. In size, diskettes range from 8-inch diskettes, as used on the IBM System/36* for example, to the 5.25-inch and more recently the 3.5-inch diskettes which are the most commonly used today. Information capacity ranges from the earliest 320KB 5.25-inch diskettes to 2.8MB 3.5-inch diskettes today. Performance is relatively slow, and coupled with the low maximum capacities, diskettes tend to be used mostly for transferring small amounts of information between computers, and sometimes for backing up information from small personal computers.

    The 5.25 and 3.5-inch standards are common to most manufacturers machines, as are certain data formats used; this means that diskettes are still one of the simplest mechanisms for exchanging information between computers of different types.


    Figure: Diskette Types - 3.5 inch diskettes are the most commonly used diskettes on the RS/6000, though the lower capacity 5.25 inch diskettes are still available.

  2. Tape Storage

    Tape storage technology has also been in existence for some time. The earliest computers would utilize tape in much the same way that disk is used today for storage of programs and data. The main difference between tape and other forms of storage device is that information is read and written sequentially. This means that random access to information on a tape is slow, as the tape must be sequentially searched from the beginning. The read/write speed can be very fast, and the information capacity very high - up to 10GB per tape - which means that tapes are currently best suited for backing up large amounts of data, or for infrequent access to archived information.

    There are two main types of tape device. Tape cartridges contain all of the tape inside a case which is inserted into a drive in much the same way as a diskette. Tape reels contain the tape wrapped around a single spool, and tend to require more complex mounting operations.

    There are many different sizes and types of tape device, as well as an equal variety of recording formats. This means that tapes are not as easily interchangeable as diskettes.


    Figure: Tape Device Types - These are some of the different tape devices available for use on the RS/6000.

  3. Disk Storage

    Magnetic disk, or Direct Access Storage Devices (DASD), are similar to diskette in design. Also known as fixed disk, magnetic disk is not however generally removable, and allows much higher read/write speeds and information capacity. Single units are now capable of storing up to 2GB and the technology is improving all the time. Disk also allows direct access to information, and coupled with the capacity and performance, makes an ideal device for loading information to, and saving from main memory.

    There are models of disk drive that do allow the removal of the internal fixed disks. The removed disk packs can then be transferred to and utilized by other computers, although due to the proprietary nature of most disk information organization, compatibility is usually only ensured between computers of the same model.


    Figure: Disk Device Types - These are some of the different disk devices available for external attachment to the RS/6000.

  4. Optical Storage

    A relatively recent technology, optical storage is based on Compact Disc technology, though there are several different mechanisms and formats in use. The simplest utilizes standard CD technology providing a Read Only Memory (ROM) capability, surprisingly known as CD-ROM. Later, more complex evolutions allow the disk to be written to once and then read from as a normal CD-ROM; this is known as Write Once, Read Many (WORM). The latest products allow complete read write capability.

    All of these products allow direct access to information, though the access time is somewhat slower than for DASD. The capacities range from 640MB for a CD-ROM to several GB in the latest products. Optical storage products are improving all the time, and the latest products are best utilized as secondary storage for less frequently accessed information, either working in parallel with DASD, or as part of a storage management system. Many optical devices allow for removable media, which make them ideal for software distribution on CD-ROM; the extraordinarily long life of information recorded on optical media (up to one hundred years) also makes this a good medium for information archive.


    Figure: Optical Device Type

This covers the various storage media available in overview. These devices also require software to drive and utilize them to their fullest potential, as well as hardware attachment methods. Both these topics will be discussed later in this chapter.

Software Concepts

So far, the purpose of the various hardware elements comprising a storage subsystem, as well as their place in the overall scheme have been discussed. In order to make these devices perform, some form of software is required to drive them. In actuality, several levels of software products are involved, and this section outlines the hierarchy.

  1. Operating System

    As is fairly common knowledge, the key player in a computer system is the operating system. This complex piece of software is responsible for making the resources of the computer available to applications in a reasonably fair and effective manner. Looking solely from the perspective of storage, the first component involved is the device driver. This piece of software is written specifically for the hardware device it provides an interface to. Essentially, it understands how to talk to the device and obtain the best performance from it. When an application wishes to communicate with the device (read or write some information), the request is made ultimately to the device driver which manages the device and executes the required function. Applications can communicate directly with the device driver, known as raw device handling, or through an intermediate software product which usually provides additional capabilities. The commands made to a device driver are usually standard (read, write, control commands) so applications need not be aware of differences in the hardware devices they are using. This does mean that a specific device driver must be provided for every device.

    Utilizing device drivers to access the devices, a number of other operating system components provide useful functions.

    These components will be discussed in more detail in Operating System Software Components.

  2. Higher Level Tools

    Higher level tools are generally applications that are designed to provide more complex storage management functions such as scheduled backup of files, disk space management, and data archive for example. These tools will usually employ many of the operating system functions to provide a more convenient interface to managing storage, which means that some of the capabilities of these products can be achieved with a good knowledge of the lower level operating system functions. Although beyond the scope of this book, some of these tools will be discussed in outline in Higher Level Storage Management Products.

  3. Applications

    Most applications will require access to information as part of their function. Many of them will access files through the file systems mentioned earlier, thus gaining the benefits of the more complex functions provided by this part of the operating system. Some applications will use storage devices directly through device drivers, which while being more complex in implementation, allows a more flexible approach to the management of their information. Databases are typical examples of applications that access storage devices in this way.




Figure: Storage Software Organization - The various levels of software used in storage subsystems make use of lower levels of software as well as sometimes utilizing the hardware directly.

Storage Management

So far, the computing environment in general has been described in order to allow the storage elements of the system to be positioned and discussed generally. This section will focus on storage management; what are the issues that this area addresses, and what aspects of a storage subsystem does it focus on.

Hardware Management

From the hardware point of view, all of the devices that can constitute a storage subsystem have been briefly discussed, and will be explored in more detail in the next chapter. The intention here is to look at what aspects of their operation are critical to overall system operation, and therefore form the focus of storage management. This should put into context the discussion in later chapters of the operating system commands and higher level tools available.

There are three main considerations, performance, availability, and capacity.

Performance

Performance is usually all about providing access to a resource such that particular criteria are met. The resources in a storage subsystem all have different characteristics and intended uses, and therefore the criteria applied are also different. In general though, it is safe to say that performance in storage products is about maximizing the throughput of information to and from the device.

As has been said, there are different criteria for each device and some examples follow, though a more complete discussion of maximizing performance for storage subsystems can be found in Storage Subsystem Design and General AIX Storage Management.




Figure: Importance of Performance Management - Users can become a trifle irritable if information is not quickly forthcoming from their computer systems.

Availability

Availability concerns designing the storage subsystem to minimize the effects of failure in any of the elements. Every environment will have different requirements in this area, but essentially the intention is to ensure the continued operation of the system despite failure in certain components. The level of redundancy, or replication of devices for replacement purposes in the event of failure, is again a trade-off with price and performance.




Figure: Importance of Availability - However carefully managed a computer system is, there will always be unforeseen circumstances when information is lost.

Capacity

The last main consideration is that of the capacity of the devices. Capacity is generally related to performance, the more space that a device has, the longer the average access time will be. This tends to be more important for devices that will be used to access data for interactive use (such as disk or optical), and it can sometimes be more prudent to utilize more lower capacity devices than fewer larger capacity devices. This will be a trade-off between cost and performance again, and there are other solutions to increasing performance through parallel access to devices.

In the main, increasing capacity with disk devices involves purchasing either larger, or more devices. With optical and tape devices there is another option, and that is the library. Optical and tape libraries provide the capability to store many tapes or optical cartridges within a managed library, such that when a request for a particular piece of data arrives, the library knows which tape or optical cartridge the information is on, and can then utilize robotics to select the item and load it into the tape or optical device. Libraries are discussed in more detail in Selecting the Correct Tape Storage Devices and Selecting the Correct Optical Storage Devices.

Thus the three major criteria from a hardware point of view, are the performance of the storage subsystem, or more generally, it's throughput, the maintenance of the required level of availability of the information stored, and the quantity of data that can be kept, or capacity.

Software Management

From the point of view of software in storage management, the main elements have been described in overview, and will be covered in detail in the section on Operating System Software Components. The intent of this section is to examine the main issues that software management aims to address, in order to provide a context for the discussion of the features currently available for storage management, as well as the new features provided in AIX Version 4.

There are three considerations, space, recovery, and administration.

Space

As has been discussed, in any computer system, there is a finite amount of fast main system memory. This, coupled with the requirement for non-volatile storage, leads to the necessity for the provision of cheaper, auxiliary storage. It is a truism to say that anything grows to fill the available resource, and this is particularly true of information stored in computer systems; thus the cheaper disk storage devices used for accessing frequently used data will also become full over time, particularly when availability designs are taken into account. Whilst cheap, disk devices are not that cheap, and so some sort of plan for managing available disk space must be developed.

Managing disk space usually involves moving less frequently accessed information out to slower, larger capacity, and cheaper per unit of information, storage devices. Optical storage devices supporting read and write capability are an ideal medium for this less frequently accessed information. Optical storage can be treated in the same way as disk, with only access time being slower. Using statistics on data access, less frequently used information can be moved to the optical media, the slightly longer delay in retrieval being acceptable for this type of information.

There is also a large amount of information that is very infrequently, if ever, accessed; tax information must be kept for five years for example, in case an inspection is required. This kind of information can be moved to even slower, massive capacity devices, such as tape libraries.

Again, this process is a trade-off between space and access time, and is usually called data archiving. Policies can be defined, and operating system tools used to manage disk space, as will be shown in the chapters on operating system software. There are also higher level tools designed specifically to provide automatic management, and some examples of these can be found in Higher Level Storage Management Products.


Figure: Space Management - Organization of the available storage space on the computer system is very important to ensure sufficient room for all of the currently required information, as well as projected growth.

Recovery

The second consideration is concerned with making provisions for failures in the storage subsystem. Disk devices can fail for a variety of reasons including mechanical faults such as head crashes, electronic failures, and corrupted data on the disk itself; optical storage can suffer from mechanical problems, tape devices can fail, and information written to tape can be unreadable due to problems with the tape media. Users of the system are also prone, on occasion, to accidentally erase vital components of the system, operating system files, or data. There is also the possibility of natural disasters such as fires, floods, or even lightning.

When a failure occurs, and it will, it would be fairly useful to be able to restore the system to its state prior to the problem. This is what recovery management is all about. There are several different strategies that can be employed, and these will be discussed in detail in the chapters on storage management, but the main point is to ensure that a current copy of the information stored in the system is available to reload from. This copy is usually stored on tape, and its currency reflects the amount of data that an organization can afford to lose. For example, if losing more than a days worth of information would be fatal to a company, then copies of at least the vital data need to be made daily. There are various ways to minimize the amount of information that needs to be copied each time, as well as high level tools to assist in managing the process. Some of the high level tools will be briefly discussed in Higher Level Storage Management Products. Again, the operating system provides the basic tools to manage this process, and this area will be examined in more detail in the chapters on storage management.


Figure: Recovery Management - The effects of disaster can be minimized if sensible backup precautions have been taken. Having the right tools and procedures for the job eases the task of recovery.

Administration

In order to make use of the devices constituting a storage subsystem, the operating system needs to made aware of them, their capabilities, and how they are to be used. Furthermore, in the event of failure, or for general maintenance, there are tasks that need doing, such as making devices temporarily unavailable so that they can be replaced, or reconfigured. Additionally, performance and usage statistics may need to be gathered so that informed planning and management can take place.

The operating system therefore provides administrative commands and tools that enable these processes to be performed. Devices can be defined to the system, made available or unavailable, configured, and monitored for performance and usage. One other useful administrative tool gives the ability to define and manage quotas for disk usage, thereby allowing a modicum of control to be exercised over usage, and thereby ease the task of managing the subsystem. These activities are examined in more detail in the chapters on operating system software components and storage management.

Diskette devices have not been mentioned in this section on storage management, mainly because their major usage is for simple transfer of small amounts of information between computer systems that are not connected via a network. In the past, when the quantity of information stored on computers (and still today for some smaller personal computers), diskette devices were used for backup purposes. Whilst remaining very inexpensive per diskette, and although the capacity has grown to several megabytes, the sheer volume of information contained in a general system backup precludes the use of diskettes for this purpose.

Summary

This chapter has looked at the basic concepts of storage management.

The first section examined the rationale behind storage subsystems, explaining at a high level, the reasons why auxiliary storage is required. The components that comprise storage subsystems:

were looked at individually, and their basic capabilities discussed. Finally, the software components used with storage subsystems were discussed: again at a high level, in order to demonstrate the levels of function provided by the various parts.

The second section looked at the rationale behind storage management. The considerations involved in hardware management:

were discussed, along with an examination of the impact that particular devices have in these areas. The issues addressed by software management: were explained in overview to put into context discussion in later chapters of these processes, and the tools and commands that enable them.