Notes 4:

Secondary Memory & Software Engineering

Why secondary memory?

Why secondary memory access during execution?:

1. Files are often larger than available main memory space
2. File systems (directories, applications, etc.) are typically larger than main memory space.
3. Even if a single user could decide what to load before execution, a multi-user, multi-processing, or multi- programming environment will be faced with competing requests by different users/processes/programs.
4. Secondary storage media are persistent.
5. Many secondary storage media are physically portable.
6. Secondary storage allows an additional level of security (both backup and fault-tolerance on the one hand, and prevention of deliberate destruction on the other).

Components of a secondary memory system:

1. A device driver (typically memory-mapped, and typically loaded when the machine is booted, or by an explicit command).
2. When a secondary memory file is active:
   • A file handle and one or more buffers in main memory.
   • The file handle typically includes the address of the driver, the address of the file in secondary memory, the type and length of the file, and the address of its buffer(s).
3. An access device for reading and writing the file.
4. A storage medium (e.g., a disk or a tape).

One More Overview of File Organizations:

Sequential:

• Access only in sequence.
• Not all records need to be used or modified.
• Can be either key-ordered or unordered (with the usual effect on linear search).
• Insert, delete, modify occur during a single sequential-access pass.

Direct:

• Quick access to any record by key value.
• Need not provide ordered access at all.
• Usually provides for cheap insert and modify; may not provide efficient delete.

Indexed sequential:

• Reasonable-cost direct and sequential access by key value.
• Also provides reasonable-cost insert, delete, modify.

Multikey:

• Allow access by more than one key.
• Efficiency is relative to an allowable set of queries.

Complexity comparisons (approximate and intuitive):

• TABLE: Costs for a file of n records:

One More Overview of Device/File/Application Characteristics:

Device characteristics:

• Space characteristics:
  • relative cost (per byte)
  • capacity (bytes).
• Time characteristics:
  • seek time
  • activation time
  • latency (time from activation to transfer)
  • data transfer rate.
• Other:
  • portability (physical, machine-to-machine)
  • organizations and access supported
  • addressability.

Application characteristics:

• Access required
  • response time
  • batch vs. interactive
  • single vs. multi-user.
• File size & record size.

Purpose of space calculations:

1. Specification of file and buffer characteristics.
2. Comparison of media (also uses timing properties).
3. Will a given file fit:
   • In available space?
   • In a single connected region?
   • In a single volume?

Deciding on a file organization:

1. Application characteristics:
   • Keys and queries required
   • Record and file size and expected changes
   • Response time and interactivity requirements
   • Activity, volatility, reference frequency (update/retrieval collisions)
   • Locality of reference properties
2. System characteristics:
   • File organizations supported by operating system
   • Languages supported and their file interfaces
   • External devices and channels available
   • Internal memory and other resources available
3. Other considerations:
   • Single or multi-user system?
   • Multiprocessing or multiprogramming supported?
   • What priority will file operation have?