Index of Documentation for People Interested in Writing and/or Understanding the Linux Kernel. Juan-Mariano de Goyeneche < jmseyas@dit.upm.es> /* * The latest version of this document may be found at: * http://www.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html */ The need for a document like this one became apparent in the linux-kernel mailing list as the same questions, asking for pointers to information, appeared again and again. Fortunately, as more and more people get to GNU/Linux, more and more get interested in the Kernel. But reading the sources is not always enough. It is easy to understand the code, but miss the concepts, the philosophy and design decisions behind this code. Unfortunately, not many documents are available for beginners to start. And, even if they exist, there was no "well-known" place which kept track of them. These lines try to cover this lack. All documents available on line known by the author are listed, while some reference books are also mentioned. PLEASE, if you know any paper not listed here or write a new document, send me an e-mail, and I'll include a reference to it here. Any corrections, ideas or comments are also welcomed. The papers that follow are listed in no particular order. All are cataloged with the following fields: the document's "Title", the "Author"/s, the "URL" where they can be found, some "Keywords" helpful when searching for specific topics, and a brief "Description" of the Document. Enjoy! ON-LINE DOCS: * Title: "The Linux Kernel" Author: David A. Rusling. URL: http://sunsite.unc.edu/linux/LDP/tlk/tlk.html Keywords: everything!, book. Description: On line, 200 pages book describing most aspects of the Linux Kernel. Probably, the first reference for beginners. Lots of illustrations explaining data structures use and relationships in the purest Richard W. Stevens' style. Contents: "1.-Hardware Basics, 2.-Software Basics, 3.-Memory Management, 4.-Processes, 5.-Interprocess Communication Mechanisms, 6.-PCI, 7.-Interrupts and Interrupt Handling, 8.-Device Drivers, 9.-The File system, 10.-Networks, 11.-Kernel Mechanisms, 12.-Modules, 13.-The Linux Kernel Sources, A.-Linux Data Structures, B.-The Alpha AXP Processor, C.-Useful Web and FTP Sites, D.-The GNU General Public License, Glossary". In short: a must have. * Title: "The Linux Kernel Hackers' Guide" Author: Michael K.Johnson and others. URL: http://khg.redhat.com/HyperNews/get/khg.html Keywords: everything! Description: No more Postscript book-like version. Only HTML now. Many people have contributed. The interface is similar to web available mailing lists archives. You can find some articles and then some mails asking questions about them and/or complementing previous contributions. A little bit anarchic in this aspect, but with some valuable information in some cases. * Title: "Conceptual Architecture of the Linux Kernel" Author: Ivan T. Bowman. URL: http://plg.uwaterloo.ca/~itbowman/papers/CS746G-a1.html Keywords: conceptual software arquitecture, extracted design, reverse engineering, system structure. Description: Conceptual software arquitecture of the Linux kernel, automatically extracted from the source code. Very detailed. Good figures. Gives good overall kernel understanding. * Title: "Concrete Architecture of the Linux Kernel" Author: Ivan T. Bowman, Saheem Siddiqi, and Meyer C. Tanuan. URL: http://plg.uwaterloo.ca/~itbowman/papers/CS746G-a2.html Keywords: concrete arquitecture, extracted design, reverse engineering, system structure, dependencies. Description: Concrete arquitecture of the Linux kernel, automatically extracted from the source code. Very detailed. Good figures. Gives good overall kernel understanding. This papers focus on lower details than its predecessor (files, variables...). * Title: "Linux as a Case Study: Its Extracted Software Architecture" Author: Ivan T. Bowman, Richard C. Holt and Neil V. Brewster. URL: http://plg.uwaterloo.ca/~itbowman/papers/linuxcase.html Keywords: software architecture, architecture recovery, redocumentation. Description: Paper appeared at ICSE'99, Los Angeles, May 16-22, 1999. A mixture of the previous two documents from the same author. * Title: "Overview of the Virtual File System" Author: Richard Gooch. URL: http://www.atnf.csiro.au/~rgooch/linux/vfs.txt Keywords: VFS, File System, mounting filesystems, opening files, dentries, dcache. Description: Brief introduction to the Linux Virtual File System. What is it, how it works, operations taken when opening a file or mounting a file system and description of important data structures explaining the purpose of each of their entries. * Title: "The Linux RAID-1, 4, 5 Code" Author: Ingo Molnar, Gadi Oxman and Miguel de Icaza. URL: http://www.ssc.com/lj/issue44/2391.html Keywords: RAID, MD driver. Description: Linux Journal Kernel Korner article. Here is it's abstract: "A description of the implementation of the RAID-1, RAID-4 and RAID-5 personalities of the MD device driver in the Linux kernel, providing users with high performance and reliable, secondary-storage capability using software". * Title: "Dynamic Kernels: Modularized Device Drivers" Author: Alessandro Rubini. URL: http://www.ssc.com/lj/issue23/1219.html Keywords: device driver, module, loading/unloading modules, allocating resources. Description: Linux Journal Kernel Korner article. Here is it's abstract: "This is the first of a series of four articles co-authored by Alessandro Rubini and Georg Zezchwitz which present a practical approach to writing Linux device drivers as kernel loadable modules. This installment presents an introduction to the topic, preparing the reader to understand next month's installment". * Title: "Dynamic Kernels: Discovery" Author: Alessandro Rubini. URL: http://www.ssc.com/lj/issue24/kk24.html Keywords: character driver, init_module, clean_up module, autodetection, mayor number, minor number, file operations, open(), close(). Description: Linux Journal Kernel Korner article. Here is it's abstract: "This article, the second of four, introduces part of the actual code to create custom module implementing a character device driver. It describes the code for module initialization and cleanup, as well as the open() and close() system calls". * Title: "The Devil's in the Details" Author: Georg v. Zezschwitz and Alessandro Rubini. URL: http://www.ssc.com/lj/issue25/kk25.html Keywords: read(), write(), select(), ioctl(), blocking/non blocking mode, interrupt handler. Description: Linux Journal Kernel Korner article. Here is it's abstract: "This article, the third of four on writing character device drivers, introduces concepts of reading, writing, and using ioctl-calls". * Title: "Dissecting Interrupts and Browsing DMA" Author: Alessandro Rubini and Georg v. Zezschwitz. URL: http://www.ssc.com/lj/issue26/interrupt.html Keywords: interrupts, irqs, DMA, bottom halves, task queues. Description: Linux Journal Kernel Korner article. Here is it's abstract: "This is the fourth in a series of articles about writing character device drivers as loadable kernel modules. This month, we further investigate the field of interrupt handling. Though it is conceptually simple, practical limitations and constraints make this an ``interesting'' part of device driver writing, and several different facilities have been provided for different situations. We also investigate the complex topic of DMA". * Title: "Device Drivers Concluded" Author: Georg v. Zezschwitz. URL: http://www2.linuxjournal.com/lj-issues/issue28/1287.html Keywords: address spaces, pages, pagination, page management, demand loading, swapping, memory protection, memory mapping, mmap, virtual memory areas (VMAs), vremap, PCI. Description: Finally, the above turned out into a five articles series. This latest one's introduction reads: "This is the last of five articles about character device drivers. In this final section, Georg deals with memory mapping devices, beginning with an overall description of the Linux memory management concepts". * Title: "Network Buffers And Memory Management" Author: Alan Cox. URL: http://www.ssc.com/lj/issue30/kk30.html Keywords: sk_buffs, network devices, protocol/link layer variables, network devices flags, transmit, receive, configuration, multicast. Description: Linux Journal Kernel Korner. Here is the abstract: "Writing a network device driver for Linux is fundamentally simple---most of the complexity (other than talking to the hardware) involves managing network packets in memory". * Title: "An Introduction to the Linux 1.3.x Networking Code" Author: Vipul Gupta. URL: http://anchor.cs.binghamton.edu/courses/cs628/linux-net.html Keywords: files, sk_buffs. Description: A short description of files under the net/ directory. Each file has a one or two lines paragraph description. sk_buffs explained, too, with some beautiful pictures. A little bit outdated. * Title: "Linux ioctl() Primer" Author: Vipul Gupta. URL: http://anchor.cs.binghamton.edu/courses/cs628/ioctl.html Keywords: ioctl, socket. Description: Little description and examples on the use and implementation of the ioctl() system call. A little bit biased towards sockets. * Title: "Writing Linux Device Drivers" Author: Michael K. Johnson. URL: http://www.redhat.com/~johnsonm/devices.html Keywords: files, VFS, file operations, kernel interface, character vs block devices, I/O access, hardware interrupts, DMA, access to user memory, memory allocation, timers. Description: Introductory 50-minutes (sic) tutorial on writing device drivers. 12 pages written by the same author of the "Kernel Hackers' Guide" which give a very good overview of the topic. * Title: "The Venus kernel interface" Author: Peter J. Braam. URL: http://www.coda.cs.cmu.edu/doc/html/kernel-venus-protocol.html Keywords: coda, filesystem, venus, cache manager. Description: "This document describes the communication between Venus and kernel level file system code needed for the operation of the Coda filesystem. This version document is meant to describe the current interface (version 1.0) as well as improvements we envisage". * Title: "Programming PCI-Devices under Linux" Author: Claus Schroeter. URL: ftp://ftp.llp.fu-berlin.de/pub/linux/LINUX-LAB/whitepapers/pcip.ps .gz Keywords: PCI, device, busmastering. Description: 6 pages tutorial on PCI programming under Linux. Gives the basic concepts on the architecture of the PCI subsystem, as long as basic functions and macros to read/write the devices and perform busmastering. * Title: "Writing Character Device Driver for Linux" Author: R. Baruch and C. Schroeter. URL: ftp://ftp.llp.fu-berlin.de/pub/linux/LINUX-LAB/whitepapers/drivers .ps.gz Keywords: character device drivers, I/O, signals, DMA, accesing ports in user space, kernel environment. Description: 68 pages paper on writing character drivers. A little bit old (1.993, 1.994) although still useful. * Title: "Design and Implementation of the Second Extended Filesystem" Author: Rémy Card, Theodore Ts'o, Stephen Tweedie. URL: http://web.mit.edu/tytso/www/linux/ext2intro.html Keywords: ext2, linux fs history, inode, directory, link, devices, VFS, physical structure, performance, benchmarks, ext2fs library, ext2fs tools, e2fsck. Description: Paper written by three of the top ext2 hackers. Covers Linux filesystems history, ext2 motivation, ext2 features, design, physical structure on disk, performance, benchmarks, e2fsck's passes description... A must read! Notes: This paper was first published in the Proceedings of the First Dutch International Symposium on Linux, ISBN 90-367-0385-9. * Title: "The Second Extended Filesystem" Author: Matthew Wilcox. URL: http://pocket.fluff.org/~mrw/linux/ext2.txt Keywords: ext2, filesystem. Description: Description of ext2's blocks, directories, inodes... Notes: Seems to be DOWN. Anyone knows another link for it? * Title: "Analysis of the Ext2fs structure" Author: Louis-Dominique Dubeau. URL: http://step.polymtl.ca/~ldd/ext2fs/ext2fs_toc.html Keywords: ext2, filesystem, ext2fs. Description: Description of ext2's blocks, directories, inodes, bitmaps, invariants ... * Title: "Journaling the Linux ext2fs Filesystem" Author: Stephen C. Tweedie. URL: ftp://ftp.uk.linux.org:/pub/linux/sct/fs/jfs/journal-design.ps.gz Keywords: ext3, journalist. Description: Excellent 8-pages paper explaining the journaling capabilities added to ext2 by the author, showing different problems faced and the alternatives chosen. * Title: "Kernel API changes from 2.0 to 2.2" Author: Richard Gooch. URL: http://www.atnf.csiro.au/~rgooch/linux/docs/porting-to-2.2.html Keywords: 2.2, changes. Description: Kernel functions/structures/variables which changed from 2.0.x to 2.2.x. * Title: "Kernel API changes from 2.2 to 2.3" Author: Richard Gooch. URL: http://www.atnf.csiro.au/~rgooch/linux/docs/porting-to-2.3.html Keywords: 2.3, changes. Description: Kernel functions/structures/variables which changed from 2.2.x to 2.3.x. * Title: "Linux Kernel Module Programming Guide" Author: Ori Pomerantz. URL: http://www.linuxdoc.org/LDP/lkmpg/mpg.html Keywords: modules, GPL book, /proc, ioctls, system calls, interrupt handlers . Description: Very nice 92 pages GPL book on the topic of modules programming. Lots of examples. * Title: "Device File System (devfs) Overview" Author: Richard Gooch. URL: http://www.atnf.csiro.au/~rgooch/linux/docs/devfs.txt Keywords: filesystem, /dev, devfs, dynamic devices, major/minor allocation, device management. Description: Document describing Richard Gooch's controversial devfs, which allows for dynamic devices, only shows present devices in /dev, gets rid of major/minor numbers allocation problems, and allows for hundreds of identical devices (which some USB systems might demand soon). * Title: "I/O Event Handling Under Linux" Author: Richard Gooch. URL: http://www.atnf.csiro.au/~rgooch/linux/docs/io-events.html Keywords: IO, I/O, select(2), poll(2), FDs, aio_read(2), readiness event queues. Description: From the Introduction: "I/O Event handling is about how your Operating System allows you to manage a large number of open files (file descriptors in UNIX/POSIX, or FDs) in your application. You want the OS to notify you when FDs become active (have data ready to be read or are ready for writing). Ideally you want a mechanism that is scalable. This means a large number of inactive FDs cost very little in memory and CPU time to manage". * Title: "The Kernel Hacking HOWTO" Author: Various Talented People, and Rusty. URL: http://www.samba.org/~netfilter/kernel-hacking-HOWTO.html Keywords: HOWTO, kernel contexts, deadlock, locking, modules, symbols, return conventions. Description: From the Introduction: "Please understand that I never wanted to write this document, being grossly underqualified, but I always wanted to read it, and this was the only way. I simply explain some best practices, and give reading entry-points into the kernel sources. I avoid implementation details: that's what the code is for, and I ignore whole tracts of useful routines. This document assumes familiarity with C, and an understanding of what the kernel is, and how it is used. It was originally written for the 2.3 kernels, but nearly all of it applies to 2.2 too; 2.0 is slightly different. ". * Title: "ALSA 0.5.0 Developer documentation" Author: Stephan 'Jumpy' Bartels . URL: http://www.math.TU-Berlin.de/~sbartels/alsa/ Keywords: ALSA, sound, soundcard, driver, lowlevel, hardware. Description: Advanced Linux Sound Architecture for developers, both at kernel and user-level sides. Work in progress. ALSA is supposed to be Linux's next generation sound architecture. * Title: "Programming Guide for Linux USB Device Drivers" Author: Detlef Fliegl. URL: http://usb.in.tum.de/usbdoc/ Keywords: USB, universal serial bus. Description: A must-read. From the Preface: "This document should give detailed information about the current state of the USB subsystem and its API for USB device drivers. The first section will deal with the basics of USB devices. You will learn about different types of devices and their properties. Going into detail you will see how USB devices communicate on the bus. The second section gives an overview of the Linux USB subsystem [2] and the device driver framework. Then the API and its data structures will be explained step by step. The last section of this document contains a reference of all API calls and their return codes". Notes: Beware: the main page states: "This document may not be published, printed or used in excerpts without explicit permission of the author". Fortunately, it may still be read... * Title: "Tour Of the Linux Kernel Source" Author: Vijo Cherian. URL: http://www.geocities.com/vijoc/tolks/tolks.html Keywords: . Description: A classic of this page! Was lost for a while and is back again. Thanks Vijo! TOLKS: the name says it all. A tour of the sources, describing directories, files, variables, data structures... It covers general stuff, device drivers, filesystems, IPC and Networking Code. * Title: "Linux Kernel Mailing List Glossary" Author: John Levon. URL: http://www.movement.uklinux.net/glossary.html Keywords: glossary, terms, linux-kernel. Description: From the introduction: "This glossary is intended as a brief description of some of the acronyms and terms you may hear during discussion of the Linux kernel". * Title: "Linux Kernel Locking HOWTO" Author: Various Talented People, and Rusty. URL: http://netfilter.kernelnotes.org/unreliable-guides/kernel-locking- HOWTO.html Keywords: locks, locking, spinlock, semaphore, atomic, race condition, bottom halves, tasklets, softirqs. Description: The title says it all: document describing the locking system in the Linux Kernel either in uniprocessor or SMP systems. Notes: "It was originally written for the later (>2.3.47) 2.3 kernels, but most of it applies to 2.2 too; 2.0 is slightly different". Freely redistributable under the conditions of the GNU General Public License. BOOKS: (Not on-line) * Title: "Linux Device Drivers" Author: Alessandro Rubini. Publisher: O'Reilly &Associates. Date: 1998. ISBN: 1-56592-292-1 * Title: "Linux Kernel Internals" Author: Michael Beck. Publisher: Addison-Wesley. Date: 1997. ISBN: 0-201-33143-8 (second edition) * Title: "The Design of the UNIX Operating System" Author: Maurice J. Bach. Publisher: Prentice Hall. Date: 1986. Pages: 471. ISBN: 0-13-201757-1 * Title: "The Design and Implementation of the 4.3 BSD UNIX Operating System" Author: Samuel J. Leffler, Marshall Kirk McKusick, Michael J. Karels, John S. Quarterman. Publisher: Addison-Wesley. Date: 1989 (reprinted with corrections on October, 1990). ISBN: 0-201-06196-1 * Title: "The Design and Implementation of the 4.4 BSD UNIX Operating System" Author: Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, John S. Quarterman. Publisher: Addison-Wesley. Date: 1996. ISBN: 0-201-54979-4 * Title: "Programmation Linux 2.0 API systeme et fonctionnement du noyau" Author: Remy Card, Eric Dumas, Franck Mevel. Publisher: Eyrolles. Date: 1997. Pages: 520. ISBN: 2-212-08932-5 Notes: French. * Title: "The Linux Kernel Book" Author: Remy Card, Eric Dumas, Franck Mevel. Publisher: John Wiley & Sons. Date: 1998. ISBN: 0-471-98141-9 Notes: English translation. * Title: "Linux 2.0" Author: Remy Card, Eric Dumas, Franck Mevel. Publisher: Gestión 2000. Date: 1997. Pages: 501. ISBN: 8-480-88208-5 Notes: Spanish translation. * Title: "Unix internals -- the new frontiers" Author: Uresh Vahalia. Publisher: Prentice Hall. Date: 1996. Pages: 600. ISBN: 0-13-101908-2 * Title: "Linux Core Kernel Commentary. Guide to Insider's Knowledge on the Core Kernel od the Linux Code" Author: Scott Maxwell. Publisher: Coriolis. Date: 1999. Pages: 592. ISBN: 1-57610-469-9 Notes: CD-ROM included. Line by line commentary of the kernel code. * Title: "Linux IP Stacks Commentary" Author: Stephen Satchell and HBJ Clifford. Publisher: Coriolis. Date: 2000. Pages: ???. ISBN: 1-57610-470-2 Notes: Line by line source code commentary book. * Title: "Programming for the real world - POSIX.4" Author: Bill O. Gallmeister. Publisher: O'Reilly & Associates, Inc.. Date: 1995. Pages: ???. ISBN: I-56592-074-0 Notes: Though not being directly about Linux, Linux aims to be POSIX. Good reference. MISCELLANEOUS: * Name: "Linux Source Driver" URL: http://lsd.linux.cz Keywords: Browsing source code. Description: "Linux Source Driver (LSD) is an application, which can make browsing source codes of Linux kernel easier than you can imagine. You can select between multiple versions of kernel (e.g. 0.01, 1.0.0, 2.0.33, 2.0.34pre13, 2.0.0, 2.1.101 etc.). With LSD you can search Linux kernel (fulltext, macros, types, functions and variables) and LSD can generate patches for you on the fly (files, directories or kernel)". * Name: "Cross-Referencing Linux" URL: http://lxr.linux.no/source/ Keywords: Browsing source code. Description: Another web-based Linux kernel source code browser. Lots of cross references to variables and functions. You can see where they are defined and where they are used. * Name: "Linux Weekly News" URL: http://lwn.net Keywords: latest kernel news. Description: The title says it all. There's a fixed kernel section summarizing developers' work, bug fixes, new features and versions produced during the week. Published every Thursday. * Name: "Kernel Traffic" URL: http://kt.linuxcare.com Keywords: linux-kernel mailing list, weekly kernel news. Description: Weekly newsletter covering the most relevant discussions of the linux-kernel mailing list. * Name: "CuTTiNG.eDGe.LiNuX" URL: http://edge.kernelnotes.org Keywords: changelist. Description: Site which provides the changelist for every kernel release. What's new, what's better, what's changed. Myrdraal reads the patches and describes them. Pointers to the patches are there, too. * Name: "New linux-kernel Mailing List FAQ" URL: Original site: http://www.altern.org/andrebalsa/doc/lkml-faq.html URL: U.S. mirror site: http://www.ececs.uc.edu/~rreilova/linux/lkml-faq.html Keywords: linux-kernel mailing list FAQ. Description: linux-kernel is a mailing list for developers to communicate. This FAQ builds on the previous linux-kernel mailing list FAQ maintained by Frohwalt Egerer, who no longer maintains it. Read it to see how to join the mailing list. Dozens of interesting questions regarding the list, Linux, developers (who is ...?), terms (what is...?) are answered here too. Just read it. * Name: "Linux Virtual File System" Author: Peter J. Braam. URL: http://www.coda.cs.cmu.edu/doc/talks/linuxvfs Keywords: slides, VFS, inode, superblock, dentry, dcache. Description: Set of slides, presumably from a presentation on the Linux VFS layer. Covers version 2.1.x, with dentries and the dcache. * Name: "Gary's Enciclopedia - The Linux Kernel" Author: Gary (I suppose...). URL: http://members.aa.net/~swear/pedia/kernel.html Keywords: links, not found here?. Description: Gary's Enciclopedia exists to allow the rapid finding of documentation and other information of interest to GNU/Linux users. It has about 4000 links to external pages in 150 major categories. This link is for kernel-specific links, documents, sites... Look there if you could not find here whar you were looking for. * Name: "The home page of Linux-MM" Author: The Linux-MM team. URL: http://www.linux.eu.org/Linux-MM/ Keywords: memory management, Linux-MM, mm patches, TODO, docs, mailing list. Description: Site devoted to Linux Memory Mangement development. Memory related patches, HOWTOs, links, mm developers... Don't miss it if you are interested in memory management development! * Name: "Kernel Newbies IRC Channel" URL: http://www.surriel.com/kernelnewbies.shtml Keywords: IRC, newbies, channel, asking doubts. Description: #kernelnewbies on irc.openprojects.net. From the web page: "#kernelnewbies is an IRC network dedicated to the 'newbie' kernel hacker. The audience mostly consists of people who are learning about the kernel, working on kernel projects or professional kernel hackers that want to help less seasoned kernel people. [...] #kernelnewbies is on the Open Projects IRC Network, try irc.openprojects.net or irc..openprojects.net as your server and then /join #kernelnewbies". * Name: "linux-kernel mailing list archives and search engines" URL: http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html URL: http://www.kernelnotes.org/lnxlists/linux-kernel/ Keywords: linux-kernel, archives, search. Description: Some of the linux-kernel mailing list archivers. If you have a better/another one, please let me know. _________________________________________________________________ Document last updated on Mon Apr 17 18:07:07 CEST 2000