Information theory 信息论

Developed by Claude Shannon and Norbert Wiener in the late Forties, information 　theory, or statistical communication theory, deals with the theoretical underpin-nings of a wide range of communication devices: radio, television, radar,computers, telegraphy and more. This book is an excellent introduction to the mathematics underlying the theory.
Designed for upper-level undergraduates and first-year graduate students, the book treats three major areas: analysis of channel models and proof of coding theorems (Chaps. 3, 7 and 8); study of specific coding systems (Chaps. 2, 4 and 5); and study of statistical properties of information sources (Chap. 6). Among the topics covered are noiseless coding, the discrete memoryless channel, error correcting codes, information sources, channels with memory and continuous channels.
The author has tried to keep the prerequisites to a minimum. However, students should have a knowledge of basic probability theory. Some measure and Hilbert space theory is helpful as well for the last two sections of Chapter 8, which treat time-continuous channels. An appendix summarizes the Hilbert space background and the results from the theory of stochastic processes necessary for these sections.
The appendix is not self-contained, but will serve to pinpoint some of the specific equipment needed for the analysis of time-continuous channels.

CHAPTER ONE A Measure of Information
　1.1 Introduction
　1.2 Axioms for the Uncertainty Measure
　1.3 Three Interpretations of the Uncertainty Function
　1.4 Properties of the Uncertainty Function; Joint and Conditional Uncertainty.
　1.5 The Measure of Information
　1.6 Notes and Remarks
CHAPTER Two Noiseless Coding
　2.1 Introduction
　2.2 The Problem of Unique Decipherability
　2.3 Necessary and Sufficient Conditions for the Existence of Instantaneous Codes
　2.4 Extension of the Condition to Uniquely Decipherable Codes
　2.5 The Noiseless Coding Theorem
　2.6 Construction of Optimal Codes
　2.7 Notes and Remarks
CHAPTER THREE The Discrete Memoryless Channel
　3.1 Models for Communication Channels
　3.2 The Information Processed by a Channel; Channel Capacity; Classification of Channels
　3.3 Calculation of Channel Capacity
　3.4 Decoding Schemes; the Ideal Observer
　3.5 The Fundamental Theorem
　3.6 Exponential Error Bounds
　3.7 The Weak Converse to the Fundamental Theorem
　3.8 Notes and Remarks
CHAPTER FOUR Error Correcting Codes
　4.1 Introduction; Minimum Distance Principle
　4.2 Relation between Distance and Error Correcting Properties of Codes; Hamming Bound
　4.3 Parity Check Coding
　4.4 The Application of Group Theory to Parity Check Coding
　4.5 Upper and Lower Bounds on the Error Correcting Ability of Parity Check Codes
　4.6 Parity Check Codes Are Adequate
　4.7 Precise Error Bounds for General Binary Codes
　4.8 The Strong Converse for the Binary Symmetric Channel
　4.9 Non-Binary Coding
　4.10 Notes and Remarks
CHAPTER FIVE Further Theory of Error Correcting Codes
　5.1 Feedback Shift Registers and Cyclic Codes
　5.2 General Properties of Binary Matrices and Their Cycle Sets
　5.3 Properties of Cyclic Codes
　5.4 Bose-Chaudhuri-Hocquenghem Codes
　5.5 Single Error Correcting Cyclic Codes; Automatic Decoding
　5.6 Notes and Remarks
CHAPTER SIX Information Sources
　6.1 Introduction
　6.2 A Mathematical Model for an Information Source
　6.3 Introduction to the Theory of Finite Markov Chains
　6.4 Information Sources; Uncertainty of a Source
　6.5 Order of a Source; Approximation of a General Information Source by a Source of Finite Order
　6.6 The Asymptotic Equipartition Property
　6.7 Notes and Remarks
CHAPTER SEVEN　Channels with Memory
　7.1 Introduction
　7.2 The Finite-State Channel
　7.3 The Coding Theorem for Finite State Regular Channels
　7.4 The Capacity of a General Discrete Channel; Comparison of the Weak and　Strong Converses
　7.5 Notes and Remarks
CHAPTER EIGHT Continuous Channels
　8.1 Introduction
　8.2 The Time-Discrete Gaussian Channel
　8.3 Uncertainty in the Continuous Case
　8.4 The Converse to the Coding Theorem for the Time-Discrete Gaussian Channel
　8.5 The Time-Continuous Gaussian Channel
……
Appendix
Tables of Values of-log2p and-p log2p
Solutions to Problems
References
Index