Bach Chapter 1. General Overview, Perspective of Unix Environment. Summary: history what is UNIX general overview 1.1 History ("colorful" ...) .1965 MIT, Bell Labs, GE, collaborated on Multics, Multics did not succeed. Unix borrowed from Multics. (pun on multics). e.g., file system organization, command interpreter (shell) borrowed from multics. .Unix - PDP 7, Ken Thompson/Dennis Ritchie. 1969. .1971 - PDP 11 port, 16-bit computer. 64k segments PDP-11 rise of "mini-computers" .Thompson - invented B language, evolved into C. B was interpreted, C compiled. .1973 - Unix done in C. .A.T.T. could not market, provided to Universities. Unix ancestral "tree" multics... v1 1969 v6 1976 - released to U's. v7 1978 - also to U's 3 way split 1. att 2. Bell Labs 3 BSD 1977 pwb (from v6) 1978 32V 1982 Sys III 1979 4.0 Xenix 1983 System V 1984 V.2 1984 4.2 1987 V.3 v9 1986 4.3 1988 V.4 1988 4.3 tahoe net 2 release 386BSD BSDI on 386/486 plan 9 SunOs Solaris BSD 4.4 - research disbanded 386bsd William/Lynne Jolitz BSDI - commercial product freeBSD/NetBSD UnixWare linux ... (sort of) USL bought by Novell netware release Novell sold to SCO/HP, maybe it has a home now with HP commercial vendors tout V.X compatibility, often BSD/V.3 mix joke: "V.4 high speed collision between SunOs and V.3" Mach in the past, some large parts of mach were based on BSD unix OSF (mach based) - "Openwar on Sun Foundation", IBM, others DEC was only real participant GNU Hurd project - doesn't seem to be going anywhere commercial systems of note HPUX IBM's version "aix", not so OSF DEC's OSF version free (complete) systems 386BSD - X86 based, seems defunct linux - X86 FreeBSD/NetBSD, similar slightly different focus for each academic os that are based on unix minix - AST comer - xinu milestones: .v6/v7 with shell interpreter, multitask o.s. ... on pdp-11 src released to Universities. Encouraged Berkeley e.g., .81-83 BSD version of UNIX on DEC VAX, 32-bit o.s., with TCP/IP networking with Ethernet support, support of large virtual address space, fast file system .At ATT, Unix Systems Group (USG) handled releases by 83; e.g., V.2. Divestitute of little Bells from ATT left ATT able to market UNIX. no longer free distribution from ATT. .1987, V.3 from ATT, included Ritchie streams. .more recently V.4, (more or less caught up with BSD) .Novell purchased USL. Curious that. "server" battle shaping up between Sun/Microsoft and ? with Windows NT... in Intel X86 world. note: number of computers being sold still going up in at least 3 major market places at home workstations embedded systems .BSD distributions started around 1981, funded by Darpa very influential .TCP/IP networking included for free in kernel instrumental in kicking off Internet Ethernet, token ring .large space program support, such as Franz Lisp . paging o.s. released BEFORE ATT . VAX release, people would buy 32V license from ATT, get 4.1 whatever from BSD and run it. (Even ATT) 32V was V7 on VAX. 1983. 4.2 BSD . early ports of UNIX typically BSD to some microprocessor e.g., 68010 -- many vendors did this. Then later on might switch to V.3 to get more "commercial". Given that ATT abandoned the business, in hindsight that was stupid. . BSD research group disbanded 1992 .Sun .continued in BSD tradition, since founder was Bill Joy instrumental in early BSD releases (can you say "vi")... .dominate workstation market. Many innovations... integrated networking major thrust e.g., NFS or network file system, opaque to client user. .RISC computer-leader, SPARC chip-set. Also firing up Intel based platforms. .transitioning from BSD based to V.4 based system. Solaris 2.0 .will java amount to something new? Some outstanding transition points .release of UNIX src code to universities, late 70's .divestiture of ATT, ATT got commericial, early 80's real influence here was on other vendors who released UNIX systems based on ATT stream. SCO, Xenix, etc. .Darpa funding of BSD system, early 80's cheap TCP/IP ethernet networking BSD src available for free .microprocessor based workstations, early 80's RISC chipsets UNIX billed as death of proprietary O.S., as ported to N uPs (good-bye VMS, OS/360) guess what: proprietary OS is back (windows NT) .recently? Sun moves to V.4 (but really a Sun hybrid), supports Intel chipset (but noone cares much) Novell purchases USL, then sells to SCO/HP Windows NT released and is that "death of UNIX"? or are all UNIX apps being ported to WNT? Windows NT -- the new VMS? not clear ----------------------------------------------------------- bottom line: 1. still the o.s. of choice for academic research and teaching. why? Unix and colleges: useful as often free although ATT licensing I think did damage in that regard. Students and faculty have access to src. src code is still available despite the lawyers and managers fundamental for "democracy of ideas" in computer science more books available about UNIX internals than elsewhere rate of evolution is still high 2. Usoft criticism: not one unix. Therefore they will keep tight control over WNT source. Problem is that tight control also means can't give to students. 3. certainly contrary to propaganda, 32-bit multi-processing o.s. is not exactly new (vax and/or multics) ----------------------------------------------------------- Why has Unix succeeded? .mostly written in H.L.L. .simple powerful user interface .pipes and filters, programming paradigm .hierarchical file system .byte stream files .device access consistent .multi-user .multi-process .hides machine; processes see Unix; not cpu/etc. .available with no single source of control -- allows widespread experimentation... Instructor submits that evolution of UNIX is higher than for any other o.s. environment... and this is one reason why UNIX won't die. X system available on almost all UNIXes for graphical user interface. However X is a graphics engine, not a particular user interface. Motif, OpenLook, Athena, ... as opposed to Apple/Mac or Windows on Intel X86 which are specific... What is UNIX? . no single vendor offering. usoft hopes to control windows NT expansion to prevent different versions a la BSD versus ATT with UNIX. .system call interface is one definition. "minix" is therefore a version of v7 UNIX. This is POSIX point of view. Therefore we could view windows NT as a variant form of unix (given POSIX compliance), or MACH for that matter (especially its networking component). .a programming environment. We expect these utilities ls/ps/cc/make/awk/sed/vi/emacs etc., etc., If available on pc, (MKS toolkit), then what do we call dos? "filters", shell programming. Unix can be seen as a set of programming languages: 1. awk 2. sed 3. make 4. perl 4, perl 5 5. shell scripts 6. c, gcc, g++ 7. sendmail setup file 8. debuggers (adb/new S5 debugger) 9. tcl/tk for easier X i/f 10. elisp in emacs Emphasis on using a prog. technique to solve a problem. .a particular O.S.? ha ha ha ha ha... .a certain openness about src availability, exchange over the Internet, a community of knowledge-exchange? ------------------------------------------------------------ 1.2 System Structure .traditional onion skin Point Of View applications/operating system/hardware .hardware independent, although getting the O.S. bogged down in the MMU hardware has been all too common .operating system dependent /ed/vi/grep/wc/ld/date/who/sh/nroff/cpp/comp etc invoke o.s. system calls .System V.3 had 64 system calls, 32 used frequently; .Unix o.s. == set of system calls 1.3 User Perspective File system, processes, "pipes and filters" 1.3.1 File System some attributes: .hierarchical, with files in directory as part of nested rooted tree .consistent, fairly uniform name space and set of system calls open/close/read/write/lseek/mkdir/rmdir/opendir/readdir /link/unlink file name: /a/b/c .create/delete files .dynamic growth, files can get bigger .protection .devices similar to files as possible .random access for files on disk .rooted tree hierarcy / /bin /etc .filetypes = special, regular, directory, IPC related (sockets/fifos) .pathnames = absolute, relative [directories]*/filename .byte stream paradigm for files - programs do not know how kernel stores data on files - memory mapping does exist in more modern versions of UNIX still contiquous set of bytes (mmap(2)) .directories are files too with a directory structure (series of records) .access permissions .owner/group/world .directory permissions govern creation of files .devices are files .special file. i/o semantics similar to normal files. don't usually access devices directory. .copy (e.g., cp a b) open for read creat for write copy() read data until eof write data close exit .sys calls return 0/-1. Use perror() or look at errno. See man 2 intro. .paradigm would fail if ASCII/binary files were different. .copy /dev/tty foo.c 1.3.2 Processing Environment .program - executable file .process - program executing .multi-tasking; many processes executing where each process has it's own virtual machine .system == collection of processes may be grouped by "job"; i.e., pipeline wc | ls and/or by user (all processes owned by jrb, group faculty) .processes created by system calls; i.e., fork(), . exec(), overlay one process with code for another . wait() - wait for process termination . exit() - give up processor .On unix, shell is just a process that uses fork/exec/wait .who .who& - asynchronous, no wait (at least immediately) .concept of current directory. Each process has a current directory. .shells csh/ksh/sh - tailorable 1.3.3 Building Block Primitives "pipes and filters" "to provide o.s. primitives that enable users to write small, modular programs that can serve as building bridges for more complex programs". .shell has i/o redirection ls ls > out mail < foo echo hi | mail jrb nroff -mm < in > out 2> err stdin, < stdout, > stderr, 2> .pipe, shell can glue two processes together. Also an o.s. notion (it's a system call) that the shell uses. cat list | sort | wc -l 1.4. Operating System Services .process control creation, termination, suspension, communication, allocation of heap storage, shared memory. .scheduling of processes - time sharing in preemptive round robin fashion .main memory management - virtual memory, swapping, paging. .secondary storage management - disk space allocated/freed. .device management - HIDES device/regular file differences for the most part. 1.5 Assumptions about Hardware .process execution modes, USER/KERNEL (supervisor) kernel mode for privileged instructions (halt) user mode can access own text/data but not that of. other processes unless text or data is shared. cannot access kernel text/data. kernel mode can access kernel text/data easily, and can access user text/data hardware supports mode switching .operating system tracks processes and switches between them, Note that processes run the system, the kernel is not a process but a shared library. .1.5.1 Interrupts/Exceptions .devices interrupt the system asynchronously. .kernel saves process context, determines which interrupt occured and "services" it. .device priorities are set in hardware design/cpu/interrupt handling circuitry .exceptions - various kind of cpu generated (not external) interrupts e.g., divide by zero. .traps - program initiated interrupts 1.5.2 Processor Execution Level (priority levels) .machine errors/nmi .clock .disk .network hw devices .terminals .network stack .sw interrupts (traps or system calls) .o.s. will mask out certain interrupts during critical section handling. 1.5.3 Memory Management .kernel lives in low memory .virtual machine == process; i.e., as if the only process in the system .kernel works with mmu hardware to map virtual to physical addresses A process has a file image (stored on disk) and a memory image (executing in memory). The addresses are roughly the same. Text addresses are 0-based. File image: 0 - text N - data + bss Process image: 0 - text N - data S - stack -------------------------------------------------------- D.O.S. versus UNIX AST's "monolithic" criticism last decade or so, academia has touted "distributed o.s." paradigm includes: .client/server reconstruction of o.s. services; e.g., file system server manages all fs access memory manager frees/allocates memory etc... .mach/chorus/minux are example o.s. .micro-kernel only supports interrupt management send/recv message facility process management .traditional system calls (in unix) based on sw interrupt to get into kernel .since we emphasize servers; servers need ability to be concurrent; i.e., need threads or "lightweight" processes that run in same address space .potentially easier to debug and change on the fly AST thinks unix is "monolithic"; i.e., too much code -- needs to be broken up into servers. Has a point. But: do ps -elf or ps -alx on UNIX system lots of network servers, maybe somewhat there