Machine Learning and Intelligent Data Analysis

General Introductions

• Book:
  D. MacKay; Information Theory, Inference and Learning Algorithms:
  Introduction which shows the connection between Information Theory and Machine Learning.
  Free Online
• Book:
  R. Duda, P. Hart, D. Stork; Pattern Classification
• M. Berthold, David J. Hand
  Intelligent Data Analysis, An Introduction (second edition)
  Springer Verlag 2003, ISBN 3-540-43060

Lectures

• Books:
  • S. Mendelson, A.j. Smola (Eds.)
    Advanced Leactures on Machine Learning
    ML Summer Schools 2002
    Springer Verlag, ISBN 3-540-00529-3
  • O. Bousquet, Ulrike v. Luxburg, Gunnar Rätsch (Eds.)
    Advanced Leactures on Machine Learning
    ML Summer Schools 2003
    Springer Verlag, ISBN 3-540-23122-6

Neuronal Networks

• C. Bishop; Neural Networks for pattern recognition: Well written introduction in feedforward NN from the perspective of statistical pattern recognition. Only supervised learning is discussed.
• J. Hertz, A. Krogh, R. Palmer: Introduction to the theory of neural computation:
  Older, but nevertheless, good introduction in NN. Covers different types of NNs.
• S. Haykin; Neural Networks: A Comprehensive Foundation
• M. Anthony and P. Bartlett; Neural Network Learning: Theoretical Foundation
  Mainly Theory

Kernel Methods and Support Vector Machines

• Web resources:
  • http://www.kernel-machines.org
• Books:
  • B. Schölkopf, A. Smola: Learning with Kernels (Recommendation, if you are searching a book about SVMs etc.)
  • Herbrich; Learning Kernel Classifiers: Theory and Algorithms

Gaussian Processes

• Tutorials:
  • David J C MacKay
    Introduction to Gaussian Processes
• Books:
  • Carl Edward Rasmussen, Christopher K.I. Williams
    Gaussian Processes for Machine Learning
    MIT Press 2006; ISBN: 0-262-18253-X

Reinforcement Learning

• Books:
  • Richard S. Sutton and Andrew G. Barto
    Reinforcement Learning, An Introduction
    MIT Press, 1998, ISBN: 0-262-19398-1

Graphical Models and Bayesian Networks

• Tutorials:
  • David Heckerman
    A Tutorial on Learning With Bayesian Networks
  • Kevin Murphy
    A Brief Introduction to Graphical Models and Bayesian Networks
• Books:
  • Michael Jordan (Editor)
    Learning in Graphical Models
    Kluwer Academic Publisher, 1998, ISBN: 0-262-60032-3

Evolutionary und Genetic Algorithms

• Tutorials:
  • A Genetic Algorithm Tutorial(PostScript)
• Books:
  • Ingo Rechenberg
    Evolutionsstrategie 94
    Frommann Verlag 1994, ISBN 3-7728-1642-8

Swarm Intelligence

• Tutorials:
  • V. Maniezzo, L.M. Gambardella, F. Luigi
    Ant Colony Optimization
• Books:
  • Marco Dorigo and Thomas Stützle
    Ant Colony Optimization
    MIT Press 2004; ISBN: 0-262-04219-3
  • E. Bonabeau, M. Dorigo, G. Theraulaz
    Swarm Intelligence
    Oxford University Press 1999, ISBN 0-19-513158-4

Feature Selection and Feature Extraction

Introductions:
• Isabelle Guyon, Andre Elissee
  An Introduction to Variable and Feature Selection
  Journal of Machine Learning Research 3 (2003) 1157-1182

Boosting and Leveraging

Introductions:
• R. Meir, G. Rätsch
  An Introduction to Boosting and Leveraging
  in S. Mendelson, A.j. Smola (Eds.)
  Advanced Leactures on Machine Learning
  ML Summer Schools 2002
  Springer Verlag, ISBN 3-540-00529-3

Applications

Clustering

• General:
  • Data Clustering - A Review
    Jain, Murty, Flynn
    
• Maximum Margin Clustering
  Linli Xu, James Neufeldy, Bryce Larsony, Dale Schuurmans
  
• Support Vector Clustering
  Asa Ben-Hur, David Horn, Hava T. Siegelmann, Vladimir Vapnik
  Journal of Machine Learning Research 2 (2001) 125-137
  
• Spectral Clustering:
  • On spectral clustering: Analysis and an algorithm.
    Andrew Y. Ng, Michael I. Jordan, Yair Weiss
    In Advances in Neural Information Processing Systems 14: Proceedings of the 2001.
    
  • Spectral Kernel Methods for Clustering
    Nello Cristianini, John Shawe-Taylor, Jaz Kandola
    In NIPS 14, 2002
    
• Associative Clustering
  Sinkkonen, Nikkilä, Lahti and Kaski
  ECML 2004
  
• Multi-View Clustering
  Steffen Bickel and Tobias Scheffer.
  Proceedings of the IEEE International Conference on Data Mining (ICDM), 2004.
  
• Hierarchical Clustering (agglomerative, divisive), dentrograms
  
• Competitive Learning: Neural Gas, Competitive Hebbian Learning, Self Organising Maps(Kohonan maps)
  Some Competitive Learning Methods
  Bernd Fritzke, 1997
• K-Means
  
• Fuzzy K-Means
  
• EM-Algorithmus. Not a clustering methods, but an algorithm used for estimating ML parameters of a model with hidden variables (FA, probabilistic PCA, ICA, mixture models etc.).
  see for instance in advanced lectures on machine learning (Eds: Bousquet, von Luxburg, Rätsch): Unsupervised Learning, Z. Ghahramani

Classification

Regression