Bullet Points

Bullet Points

This post lists out various topics under the machine learning subject.

Data

• data modalities
  • numbers
  • texts
  • images
  • videos
  • audios
• data cleaning
• imbalanced data
• data normalization (one pitfall)
  • standardization
• data augmentation
• data splitting
  • cross validation
  • RANSAC
• feature extraction/engineering
  • domain expertise
  • mathematical methods
    • PCA
    • kernel methods
  • neural network self-adaptation

Model

• supervised vs. unsupervised
  • supervised learning
    • fits towards given target
    • includes
      • classification
        • logistic regression
        • support vector machine
        • Bayes classifier
        • linear discriminant analysis
        • extrapolation
          • from classification model to regression model
          • from binary classification to multi-class classification
      • regression (linear and non-linear case)
  • unsupervised learning
    • discovers inherent structures within/among the data
    • includes
      • clustering
      • anomaly detection
      • dimensionality reduction
      • manifold embedding
      • latent variable modelling
        • hidden Markov model
        • conditional random field
        • independent component analysis
  • reinforcement learning
    • adapts to the environment via explore-and-exploit strategy
    • depending on the knowledge of the environment, categorizes into
      • model-based
      • model-free
    • depending on the learning objective, categorizes into
      • value-based (prediction problem)
      • policy-based (control problem)
• discriminative vs. generative
• linear vs. non-linear (depending on whether the decision boundary is a hyperplane)
• parametric vs. non-parametric (depending on whether the data is characterized 1) by a distribution 2) with a finite number of parameters)
• ensemble method
  • bootstrap aggregating
    • random forest
  • gradient boosting
    • least square boosting
    • AdaBoost
    • LogitBoost
• regularization and overfitting

Evaluation

• training criterion

  • mean squared error
  • cross entropy
  • impurity
    • Gini index
    • entropy

• testing criterion

  • confusion matrix

    Real	Positive	Negative
    Positive	TP	FN
    Negative	FP	TN

    • accuracy
    • precision
    • recall
    • F-score

  • precision-recall curve

  • receiver operation characteristics

  • mean average precision