MACHINE LEARNING

111-1 / Fall 2022

TA: 伏宇寬 (FU, YU-KUAN), 林仲偉 (LIN, ZHONG-WEI), 賴彥儒 (LAI, YEN-RU)

楊智凱 (YANG, CHIH-KAI)

WEEK 2 - 09/16

Topic Introduction; Regression; Bias and Variance Errors

Slide Video

PPT EE5184 Syllabus

PPT Introduction

PPT Regression

VIDEO What is ML

VIDEO Introduction - 1

VIDEO Introduction - 2

VIDEO Regression - 1

VIDEO Regression - 2

VIDEO Regression - 3

VIDEO Regression - 4

WEEK 3 - 09/23

TopicLinear Model Classification: Probabilistic Generative Model, Logistic Regression

Slide Video

PPT Bias and Variance

PPT Classification

PPT Logistic Regression

VIDEO Recap

VIDEO Bias and Variance - 1

VIDEO Bias and Variance - 2

VIDEO k fold corss validation

VIDEO 3 fold corss validation example

VIDEO Classification Prob Gen Model 1

VIDEO Classification Prob Gen Model 2

VIDEO Generative model prob is sigmoid

WEEK 4 - 09/30

TopicNeural Networks: Introduction, Gradient Decent and Back Propagation, Tips, Implementation

Slide Video

PPT Backpropagation

PPT Deep Learning

PPT Tips for Deep Learning

PPT Gradient Descent

VIDEO Logistic Regression - 1

VIDEO Logistic Regression - 2

VIDEO Logistic Regression - 3

VIDEO Appendix and Deep Learning

VIDEO Deep Learning - 2

VIDEO Gradient Descent - 1

VIDEO Gradient Descent - 2

WEEK 5 - 10/07

Topic1. Convolutional Neural Network (CNN) (看李宏毅教授教學影片)
    2. Dimensionality Reduction: Principle Component Analysis

Slide Video

PPT CNN

PPT PCA

PPT PCA SVD

VIDEO Principle Component Analysis - 1

VIDEO Principle Component Analysis - 2

VIDEO Diagonalizability and Symmetric Matrix

VIDEO PCA algorithm step by step

VIDEO PCA QA

VIDEO Properties of PCA - 1

VIDEO Properties of PCA - 2

VIDEO Eckart-Young-Mirsky Theorem and Low Rank Approximation update

WEEK 6 - 10/14

TopicAuto encoder

Slide Video

PPT Unsupervised Learning: Deep Auto-encoder

PPT Unsupervised Learning: Neighbor Embedding

VIDEO PCA in affine set

VIDEO Proof of Eckart-Young-Mirsky Theorem

VIDEO Explain Proof of Eckart-Young-Mirsky Theorem

VIDEO PCA as Optimal Solution

VIDEO PCA and Autoencoder

VIDEO LLE and Laplacian Eigenmap

WEEK 7 - 10/21

TopicNeighbor Embedding

Slide Video

PPT Final Project

VIDEO Final Project Announcement

WEEK 8 - 10/28

Topic1. Ensemble: Random forest, AdaBoost
    2. Recurrent Neural Network (看李宏毅教授教學影片)

Slide Video

PPT AdaBoost

PPT Ensemble

PPT Recurrent Neural Network

VIDEO Decision Tree

VIDEO Random Forest and boosting

VIDEO Adaboost

VIDEO Adaboost Example

VIDEO Adaboost Coefficient

VIDEO Gradient Boost

WEEK 9 - 11/4

TopicExpectation Maximization and Gaussian Mixture Models

Slide Video

PPT Expectation Maximization and Gaussian Mixture Models

VIDEO Maximum-Likelihood Estimation for Single Gaussian Model

VIDEO Maximum-Likelihood Estimation for Gaussian Mixture Model

VIDEO Expectation Maximation Theorem

VIDEO Why Cross Entropy Difference Equals to 0

VIDEO QA for EM Algorithm

VIDEO EM Algorithm for GMM Expectation-Step

VIDEO EM Algorithm for GMM Maximization-Step

VIDEO Appendix: EM Algorithm introduction

WEEK 10 - 11/11

TopicSemi-Supervised Learning

Slide Video

PPT Semi-supervised Learning

VIDEO Semi-supervised Learning

VIDEO Semi-supervised Generative Model

VIDEO Semi-supervised Learning Low Density Seperation

VIDEO Entropy-based and Graph-based approach

VIDEO Graph-based approach Elaborate

WEEK 11 - 11/18

TopicVariational Auto-Encoder

Slide Video

PPT Variational Auto Encoder

VIDEO Variational Auto Encoder

VIDEO ELBO of VAE

VIDEO VAE and Gaussian Mixture Model - 1

VIDEO VAE and Gaussian Mixture Model - 2

VIDEO VAE and Gaussian Mixture Model - 3

WEEK 12 - 11/25

Topic1. Support Vector Machine – Margin and primal form
    2. Duality Theory of Constrained Optimization - Introduction

Slide Video

PPT Support Vector Machine Introduction

PPT Support Vector Machine Optimization

VIDEO Linear SVM

VIDEO Soft Margin SVM

VIDEO Optimization of Soft Margin SVM

VIDEO Properties of Convex Function

VIDEO Representer Theorem of SVM

VIDEO Primal and Dual Problem of SVM

VIDEO Grometric Repesentation of Primal and Dual Problem

WEEK 13 - 12/02

Topic1. Strong Duality Theorem
    2. Support Vector Machine: Kernel form and KKT conditions

Slide Video

WEEK 14 - 12/09

TopicProbably Approximately Correct Learning

Slide Video

PPT Support Vector Machine Kernel Methods

PPT Lagrangian Duality

PDF Lagrangian Duality

VIDEO 期末提醒

VIDEO SVM and its Dual Problem and KKT conditions - 1

VIDEO SVM and its Dual Problem and KKT conditions - 2

VIDEO SVM and its Dual Problem - 1

VIDEO Lagrange Duality - 1

VIDEO Lagrange Duality - 2

VIDEO Lagrange Duality - 3

VIDEO Lagrange Duality - 4

VIDEO Lagrange Duality - 5

WEEK 15 - 12/16

TopicPrivacy Preserving Machine Learning

Slide Video

VIDEO Morris Chang Lecture - 1

VIDEO Morris Chang Lecture - 2

VIDEO Morris Chang Lecture - 3


EXERCISE

HOMEWORK

Name Question Answer

Math Problem 1

Math Problem 2

Math Problem 3

Math Problem 4

Math Problem 5

PDF Mathematic Background, ...

PDF Convergence of K-means Clustering

PDF Convolution, Batch Normalization, ...

PDF LSTM Cell, Multiclass AdaBoost

PDF Kernel, SVM, SVR

Not Available

Not Available

Not Available

Not Available

Not Available

PROGRAM

Name Slide / Video Code

Program 1

Program 2

Program 3

Program 4

Program 5

PPT PM2.5 PredictionVideo P1 video

PPT Face Expression PredictionVideo P2 video

PPT EmbeddingVideo P3 video

PPT Text Sentiment ClassificationVideo P4 video

PPT Income 50K prediction

FILE Program1 Code

FILE Program2 Code

FILE Program3 Code

FILE Program4 Code

FILE Program5 Code