System Software
System Software can be distinguished as the (1) Operating System and (2) System Utilities. From your Linux studies, recall the Kernel is the central part of an operating system as it manages the operations of the computer and the hardware. OS knowledge may be the most important knowledge you can possess since this is required for Security and you continually hear me state we think about security first.
OS Framing Introduction
Recall the basic model of computation is input => processing => output. It is the OS that controls or manages this computation as it provides a usable or abstract interface to the raw hardware. The Operating System (OS) is the intermediary software (e.g. a group of programs) that facilitates the use of the computer’s hardware/physical and software/virtual resources (abstraction => the underlying raw computer hardware is represented by higher-level conceptual constructs that are easier to use and more accessible to the user — or in simpler words, OS abstraction makes the computer easier to use – see transformation and multiplexing further below).
Abstraction – referencing something as a general quality rather than its characteristic, concrete realities, specific objects, or actual instances. In computing, it establishes an acceptable interface allowing a person to interact with the system without fully knowing or understanding the complex details of the function (e.g. access an entire mp3 file rather than the various clusters where it is stored on the HD).
OS abstraction example, recall the Fetch Execute Cycle simulation I presented. The machine architecture is awkward to work with and varies from system to system (e.g. complex instruction set, memory organization, Input / Output, bus structure, etc.). Would anyone use computers if they had to interface with the computer at that level (e.g. storage locations, memory locations, operations, operands, etc.. all represented in Binary/Hex)?
Technology Acceptance Model (TAM) => Perceived Ease of Use (PEU) vs. Perceived Usefulness (PU). This means that if something is easy to use, people will adopt and use it. If something is difficult to use its adoption will depend on how useful it is. This directly applies to Operating Systems (think of an easy GUI like Windows, Mac in contrast to command line Linux… is anyone moving from their desktop GUI to command line Linux at this point?)
Interfaces – recall there are 3 distinct computer interfaces: 1) physical, 2) software/logical, 3) human/computer (HCI). The OS manages aspects of all 3 interfaces.
Generations
Recall we studied generations of computers from a hardware perspective but I stated that there were equivalent advances in Operating Systems and Programming Languages that coincided with the generational advances in hardware. A nice quick summary of OS generations may be found here. Most recently, we have seen the emergence and quick evolution of mobile devices and their OS.
To start, please review the Boot Sequence and reread the Linux Labs & readings as is required since Linux provides us with the opportunity to interface with and apply our OS concepts. It is also worth exploring the Extensible Firmware Interface as this is the new standard: http://en.wikipedia.org/wiki/Extensible_Firmware_Interface
OS Design Goals
Performance and functionality are the keys to the entire system but note there is no precise correct way to do things, just proven methods. OS design goals are a blend of convenience and efficiency noting that convenience and efficiency are trade-offs and often negatively impact each other. As an example consider the Windows OS and its GUI which is designed for convenience and ease of use. In this regard, Windows can never be as efficient as a dedicated Linux server that is optimized for efficiency and may not even have GUI (recall the GUI must be in memory so it consumes a critical resource – e.g. memory).
System Resources can be categorized as either logical/virtual resources or as physical resources. Note this resource distinction can often be thought of in terms of logical software and physical hardware.
Logical or virtual resources include files (the actual digital information, not the physical space the file takes up in storage) and logical Internet connections.
Physical resources include the CPU (CPU processing cycles are allocated in time slices by the OS Scheduler), memory, storage, physical network connections that include Wi-Fi, Bluetooth, Ethernet network interface cards (NIC), and peripherals (e.g. I/O devices that include printers, keyboards, cameras, touch screens, etc.).
OS is the Resource Manager (provides resource protection)
First and foremost, the OS is the resource manager (both logical and physical resources). In this capacity, the OS provides system security and protection. To accomplish this the OS runs in kernel or SU or protected mode whereas applications run in user mode. This is clearly evident when we use the “sudo” command in Ubuntu. Note security problems are exacerbated by networked/distributed systems.
Why does the system need protection? Imagine if your word processor decided to write files over your MP3 collection… this would be bad so your word processor is not given this authority. What if your print jobs were interleaved with other users’ print jobs rather than printing in their entirety (e.g. print one page for you, print one page for them, print one page for you….). This would be bad or at least bothersome so some resources will require mutually exclusive access (i.e. one process has access to the resource at a time). It is the OS that maintains the file allocation table which records where all physical fragments of each logical file exist as well as what blocks are free.
System Calls
As a result of the above scenarios, the OS acts as the system’s “policeman” and protects and manages system resources. Put another way, every process must request the OS for access to restricted/protected resources using a system call therefore it is the OS that actually accesses and manages the resources. To further illustrate this scenario please understand – a word processor does not open, write to, print, or save a file. The word processor sends a request (system call) to the OS to open a file, display the file on the screen, print the file, save the file to disk, etc. using System Calls.
Note my use of the word “agent” as increasingly we have software that acts for us, especially in response to events (i.e. event-driven programming). As an example, your phone’s weather app could request the OS vibrate or make a tone if it were to receive a notification of dangerous weather in your area. Of course, the user enabled that setting but the interrupt was generated by the app’s data/preferences.
Process vs Program
A process is an active or dynamic entity (active program) as it has been allocated logical and physical resources by the OS whereas a program is a static entity sitting somewhere on storage (and this could be somewhere on a network).
Think about this… you install a program by loading it on your hard drive and you may run a program by clicking or double-clicking on its icon but once running; the program is used to create a running or dynamic process. Now I just used the term “dynamic”. In computing, dynamic means during runtime and static is not or prior to runtime.
Now recall I stated at the course outset that Computing texts are often misread by the average person since every word looks familiar (i.e. you have seen every word on this page before). If so, from the above text, what is “mutual exclusive access”? Note that I used the word “process” above rather than “program”… do you know the distinction between process and program? If not this is an indication that we need to reread the material with more precision.
Lecture Capture Part 1
Lecture Capture Part 2
For the MS Windows Utilities Lecture Capture please see below.
For the File Format Forensics Lecture Capture please see the submenu.
OS Abstraction => Transformation & Multiplexing
The computer (or machine) architecture provides the basic machine instruction set (e.g. store, move, retrieve, arithmetic, logic, etc.).
Transformation performs abstraction by creating a logical resource from a physical resource. Put another way, the OS hides the complexity of the hardware by providing agents (users/applications) with a generic or uniform logical interface. To illustrate this consider 2 different programs that can request printing/print resources from the OS in the same way even if the printers are changed and this can even be extended to the user’s interface where they print from different programs in the same way (e.g. uniform logical interfaces).
This is the convenience introduced above as agents do not want to access the computer hardware and resources using the machine’s instruction set but instead want a standard consistent high-level interface.
Note this transformation is necessary for the OS to provide a consistent base for applications. To illustrate this from a mobile perspective, app developers would not want to develop separate versions of ios Apps for the iPhone 4, 5, …. nor would Android app developers want to develop separate versions for Samsung, HTC, etc. The solution is to provide a high-level application programming interface (API).
Multiplexing extends transformation as it facilitates the creation of multiple logical resources from a single physical resource. Put another way the OS facilitates physical resource sharing by allocating and managing logical resources determining what agents (users and processes) have access and authorization to use which resources.
The OS performs space multiplexing by dividing up and allocating memory to processes and storage to users and files. The OS performs time multiplexing by giving processes CPU time slices. Also, think about your single WiFi or Ethernet connection (e.g. physical NIC) that supports multiple (multiplexed) logical connections as you can have multiple Web browsers open, your SSH connection to the HVCC AcadNX server, an email client, etc. open at the same time all multiplexed into/through a single physical network connection. Note multiplexing is a core concept in networking so we will see it again in networking
If the system has a single CPU (single-core) space multiplexing and time multiplexing facilitates multi-tasking where several processes can be in memory and the OS gives the user the appearance that several programs are running concurrently. Note that multi-processing takes this one step further but requires multiple CPUs or CPU cores so that processes can be run concurrently.
OS Components => File Manager, Memory Manager, Scheduler, Device Manager, Utilities
File System (File Manager)
The file manager implements the storage abstraction by mapping a filename to a collection of physical blocks and storage devices and uses device drivers to read and write blocks to/from particular devices. The file manager provides a spectrum of commands for file and directory management and most importantly it provides protection which is absolutely necessary for a multi-tasking, multi-user environment.
File system multiplexing and transformation provide the abstraction that links blocks of the storage system together into logical files that users and programmers can access. Now as IT professionals, we need to recall all transfers are performed in blocks (sectors/clusters) and that data must be in memory to be operated on. As an example, if data in storage needs to be modified it needs to be read into memory, processed, and written back to storage. Looking at this more discretely, when data is written to the storage device, the application’s data structure is flattened into a byte stream by the OS, and this stream is stored as a set of blocks (sectors/clusters). When data is read from a device it is read block by block converted into a stream of bytes and converted back into an application-level or OS executable data structure.
Files
As a basis, a file is a named collection of data stored on a device both the OS and applications rely on the presence of structure in the data. The file system is probably the most visible OS component to users as it is the structured mechanism by which users save information from 1 session to another. Now, of course, Apple iOS has changed the model since users cannot directly access the file system. As a Computer Scientist, this left me uncomfortable at first since I have always had access to the file system but I quickly saw the inherent security improvement as requiring all access to the file system to go through a managed iTunes interface that can improve security (i.e. Apple approves ios Apps). Again, recall my statement that security is 75% policy.
To illustrate the import of files, consider the vast majority of programs read information from a file, process the information, and write back to a file. As an example, a compiler reads a source program file, translates the program into machine code (i.e. into the OS’s executable file structure), and writes a relocatable file and a report. This model is so prevalent it is built into the UNIX/Linux process model. In Linux, at process creation time, a Linux process automatically has access to 3 files: stdin (the default references communication device), stdout (the default references communication device), and stderr (again the default references communication device).
Directories (folders)
A directory is a user-determined set of logically associated files and other directories of files. Directories are the mechanism by which humans organize sets of files as it segments the namespace (i.e. it is no longer a flat or single namespace).
RAM Disk
A RAM Disk can dramatically improve the performance of your Windows system by running applications as if they were stored in RAM (virtual hard drive). In a nutshell here is a way to get SSD HD performance even if you don’t have an SSD HD.
http://www.pcworld.com/article/260918/how_to_supercharge_your_pc_with_a_ram_disk.html
Memory Manager
Recall a program is a static entity as we install a program on storage. When the program is invoked or loaded into memory it becomes a process and the OS allocates memory to the process. This process’s memory will be further distinguished as a Code Segment, Data Segment, and Stack Segment. The OS must ensure that a process’s memory is protected from other processes.
Virtual Memory creates a larger memory space by utilizing secondary storage and the OS transfers data between main memory and secondary virtual memory as required.
You can also use a USB drive for virtual memory which is much faster than mechanical/magnetic HDs. Instructions can be found here: http://www.orclage.com/how-to-use-usb-flash-drive-as-ram-in-windows-8-7-and-xp-pendrive/
Scheduler (plus some terms)
The Scheduler allocates the CPU to processes/threads. (A thread is a light-weight process that shares address space)
Preemptive scheduling – continues until the process is interrupted (Scheduler/time slice, I/O, i.e.)
Multi-programming – A computer with more than one program in memory at a time sharing a single CPU introduced in the 3rd OS generation.
Multitasking – an extension of multi-programming that adds the concurrent or interleaved execution of two or more jobs by a single CPU introduced in the 4th OS generation. Note this can be preemptive or non-preemptive (non-preemptive requires cooperation).
Multiprocessing – extends multitasking and uses more than one CPU at a time.
Device Manager
Responsible for connecting and managing (allocate, monitor, de-allocate) all connected devices (e.g. printers, ports, disk drives, etc.). Some important concepts follow:
Programmed I/O (PIO) requires CPU involvement
Direct Memory Access (DMA) does not require the CPU as the device performs the transfer to and from memory.
Buffering overlaps a single process’s computation w its own I/O
Spooling overlaps I/O across process execution (multiple processes) by placing items in a buffer so they can be retrieved by the appropriate device when needed (can enforce mutual exclusion).
System Utilities
Note this area can be fuzzy but generally includes compilers, editors, and shell utilities (e.g. compression and disk utilities).
MS Windows 7 File Utilities, DOS & msconfig Presentation
Distributed Computing
Distributed Computing is a software system in which components located on networked computers communicate and coordinate their actions by passing messages (BTW – How is all computation performed in object-oriented systems -> by passing messages). Examples of distributed systems include Service-Oriented Architectures (SOA), massively multiplayer online games, and peer-to-peer applications (e.g. Napster, Torrents, Windows Homegroup, etc.).