Learning from Data
Concepts, Theory, and Methods
Wiley - IEEE (Band Nr. 1)
2. Auflage September 2007
560 Seiten, Hardcover
Lehrbuch
Kurzbeschreibung
An interdisciplinary framework for learning methodologies, covering statistics, neural networks, and fuzzy logic, Learning from Data provides a unified treatment of the principles and methods for learning dependencies from data. It establishes a general conceptual framework in which various learning methods from statistics, neural networks, and fuzzy logic can be applied, showing that a few fundamental principles underlie most new methods being proposed today in statistics, engineering, and computer science. Complete with over one hundred illustrations, case studies, and examples, Learning from Data pursues several goals: first, to introduce and describe recent advances in learning methods, such as recent work on Support Vector Machines and boosting; second, to better show the relationship between the VC (Vapnik-Chernovenkis) theoretical approach and other well-known statistical paradigms, such as regularization and robust statistics; third to show the connection between VC-based methodology and practical applications.
An interdisciplinary framework for learning methodologies-now revised and updated
Learning from Data provides a unified treatment of the principles and methods for learning dependencies from data. It establishes a general conceptual framework in which various learning methods from statistics, neural networks, and pattern recognition can be applied-showing that a few fundamental principles underlie most new methods being proposed today in statistics, engineering, and computer science.
Since the first edition was published, the field of data-driven learning has experienced rapid growth. This Second Edition covers these developments with a completely revised chapter on support vector machines, a new chapter on noninductive inference and alternative learning formulations, and an in-depth discussion of the VC theoretical approach as it relates to other paradigms.
Complete with over one hundred illustrations, case studies, examples, and chapter summaries, Learning from Data accommodates both beginning and advanced graduate students in engineering, computer science, and statistics. It is also indispensable for researchers and practitioners in these areas who must understand the principles and methods for learning dependencies from data.
Notation.
1. Introduction.
1.1 Learning and Statistical Estimation.
1.2 Statistical Dependency and Causality.
1.3 Characterization of Variables.
1.4 Characterization of Uncertainty.
References.
2. Problem Statement, Classical Approaches, and Adaptive Learning.
2.1 Formulation of the Learning Problem.
2.2 Classical Approaches.
2.3 Adaptive Learning: Concepts and Inductive Principles.
2.4 Summary.
References.
3. Regularization Framework.
3.1 Curse and Complexity of Dimensionality.
3.2 Function Approximation and Characterization of Complexity.
3.3 Penalization.
3.4 Model Selection (Complexity Control).
3.5 Summary.
References.
4. Statistical Learning Theory.
4.1 Conditions for Consistency and Convergence of ERM.
4.2 Growth Function and VC-Dimension.
4.3 Bounds on the Generalization.
4.4 Structural Risk Minimization.
4.5 Comparisons of Model Selection for Regression.
4.6 Measuring the VC-dimension.
4.7 Summary and Discussion.
References.
5. Nonlinear Optimization Strategies.
5.1 Stochastic Approximation Methods.
5.2 Iterative Methods.
5.3 Greedy Optimization.
5.4 Feature Selection, Optimization, and Statistical Learning Theory .
5.5 Summary.
References.
6. Methods for Data Reduction and Dimensionality Reduction.
6.1 Vector Quantization.
6.2 Dimensionality Reduction: Statistical Methods.
6.3 Dimensionality Reduction: Neural Network Methods.
6.4 Methods for Multivariate Data Analysis.
6.5 Summary.
References.
7. Methods for Regression.
7.1 Taxonomy: Dictionary versus Kernel Representation.
7.2 Linear Estimators .
7.3 Adaptive Dictionary Methods.
7.4 Adaptive Kernel Methods and Local Risk Minimization.
7.5 Empirical Studies.
7.6 Combining Predictive Models.
7.7 Summary.
References.
8. Classification.
8.1 Statistical Learning Theory Formulation.
8.2 Classical Formulation.
8.3 Methods for Classification.
8.4 Combining Methods and Boosting.
8.5 Summary.
References.
9. Support Vector Machines.
9.1 Motivation for margin-based loss.
9.2 Margin-based loss, robustness and complexity control.
9.3 Optimal Separating Hyperplane .
9.4 High-Dimensional Mapping and Inner Product Kernels.
9.5 Support Vector Machine for Classification.
9.6 Support Vector Implementations.
9.7 Support Vector Machine for Regression.
9.8 SVM Model Selection.
9.9 SVM vs regularization approach.
9.10 Single-class SVM and novelty detection.
9.11 Summary and discussion.
References.
10. Non-Inductive Inference and Alternative Learning Formulations.
10.1 Sparse High-Dimensional Data.
10.2 Transduction .
10.3 Inference Through Contradictions.
10.4 Multiple Model Estimation.
10.5 Summary.
References.
Appendix A: Review of Nonlinear Optimization.
Appendix B: Eigenvalues and Singular Value Decomposition.
Index.
Filip Mulier, PhD, has worked in the software field for the last twelve years, part of which has been spent researching, developing, and applying advanced statistical and machine learning methods. He currently holds a project management position.
