Lecture:2019/6/27-2814:00-16:00, Prof. Lin Wenwei


Title:Nonlinear Smooth Support Vector Machines I, II

Nonlinear Smooth Support Vector Machines,


Time:6/27, 14:00-16:00)


(1) Review of optimization problems with constraints

----Primal form, dual form, Karush-Kuhn-Tuker (KKT) conditions.

----Tangent vectors to feasible set and linearized feasible directions.


(2) Binary classification problems/Supervised learning problems

----Linearly separable case: Maximizing the margin between boundary planes, primal and dualforms.

----Nonseparable case: primal/dual maximization problems for 1-norm/2-norm soft margin SVM.


(3) Nonlinear support vector machine

----Two spiral data set.

----Learning linear machine in feature space.

----Kernel: represent inner product in feature space.

----Kernel Techniques: monomials of degree d, polynomial kernel, Guassian (radial basis function) kernel.

----Dual representation of SVM classifier.


(4) Smooth support vector machine

----SVM as an unconstrained minimization problem.

----Smooth with plus function.

----Newton-Armijo Algorithm.


(5) Nonlinear smooth support vector machine

----Nonlinear SSVM motivation.

----Kernel trick: Gaussian kernel, monomials, polynomials.

----Nonlinear classifier.


(6) Reduced support vector machine

----Reduced SVM: A compressed model.

----A nonlinear kernel application: checkerboard training set.

----Using 50 randomly selected points out of 1000 points.

----Compressed model vs full model.











Title:Clustering and Expectation/Maximization Algorithms I, II


(Time6/28, 14:00-16:00)



(1) Searching the optimal combination of the regularization parameter and the width parameter in the Gaussian kernel

----Grid search, Nested uniform design method (UDM).

----Experimental results: grid search vs UDM (13/9) vs UDM(9/5).


(2) Three fundamental algorithms

----Naive Bayes classifier.

----K-nearest neighbors algorithm.

----Online perception algorithm.


(3) Unsupervised learning problems

----K-Means clustering problem formulation.

----K-Means Algorithm.

----K-Means ++ Algorithm.


(4) Expectation/Maximization Algorithm

----E-step: Compute the probability, the point n is generated by distribution k.

----M-step: update mean, variance and probability distribution k.

XML 地图 | Sitemap 地图