Professional Documents
Culture Documents
Statistical Machine Learning (CSE 575) : About This Course
Statistical Machine Learning (CSE 575) : About This Course
Big Data
Learning Outcomes
yy Implement more advanced learning algorithms such as support vector machines and
convolutional neural networks
yy Design a deep network using an exemplar application to solve a specific problem
Projects
yy Project 1: TBD
yy Project 2: TBD
yy Project 2: Implement k-means algorithm and its variants on a real document clustering task
yy Project 3: Image Classification using a standard dataset to train a deep network to classify
images and evaluate the performance on both the training set and the test set
Course Content
Instruction
yy Video lectures
yy Other videos
yy Readings
yy Interactive learning objects
yy Live office hours
Assessments
yy Practice activities and quizzes (auto-graded)
yy Practice assignments (instructor- or peer-reviewed)
yy Team and/or individual project(s) (instructor-graded)
yy Midterm or final exam (proctored, graded)
Technology Requirements
Hardware
yy Standard with major OS
Course Outline
Unit 1: Introduction to Machine Learning
Learning Objectives
1.1 Describe common misconceptions of machine learning
1.2 Define machine learning
1.3 Distinguish between supervised learning and unsupervised learning
1.4 Compare numerical and graphical data representations
1.5 Describe applications of machine learning
Learning Objectives
2.1 Apply common probability distributions in machine learning applications
2.2 Use maximum likelihood estimate (MLE) for parameter estimation
Module 1: Probability
Discrete Random Variables
Probability Mass Function (PMF)
Common Distributions of PMF
• Uniform
• Binomial
Joint Probability Mass Function
Conditional Probability
Relationship Between Marginal and Joint Probability
Bayes Theorem
Independent Random Variables
Continuous Random Variables
Probability Density Function (PDF)
Common Distributions of PDF
• Normal
• Beta
Joint Probability Density Function
Moments of Random Variables
Learning Objectives
3.1 Differentiate between generative and discriminative models for supervised learning
3.2 Implement fundamental learning algorithms such as Naive Bayes and Logistic Regression
3.3 Interpret empirical comparisons of Naive Bayes and Logistic Regression
Learning Objectives
4.1 Differentiate between linearly separable and non-separable support vector machines
4.2 Explain the role of the kernel trick in support vector machines
4.3 Explain options for picking magic parameters in support vector machines
4.4 Implement the more advanced learning algorithm known as support vector machines
Module 2: Separable
Linearly Separable Example
Max-margin Separating Hyperplane
Margin Maximization with Canonical Hyperplanes
Optimization Problem of SVM: separable case
Dual SVM Formulation: separable case
Module 3: Non-separable
Linearly Non-separable Example
Hinge Loss
Optimization Problem of SVM: non-separable case
Dual SVM Formulation: non-separable case
Input Space to Feature Space
Kernel Trick
• Common Kernels
• Test Example
• SVM with the Kernel Trick
Learning Objectives
5.1 Differentiate between clustering in supervised vs. unsupervised learning
5.2 Explain how to efficiently cluster data
5.3 Apply the k-means algorithm
5.4 Explain the relationship between the several K-means variants
Module 2: K-means
K-means for clustering
K-means models
Properties of the K-means algorithm
• Initialization
• fix the cluster clusters, find the cluster membership
• fix the cluster membership, update the cluster center
• Repeat the above two steps until convergence
Comparing K-means clusterings
A Numerical Example
• Input data, plot them in 1-d space
• Pick the initial cluster centers
• Run k-means algorithm one iteration
• Show how the cluster membership changes
• Show how the cluster centrs change
K-means algorithm considerations
Learning Objectives
6.1 Illustrate the process of dimensionality reduction
6.2 Apply the PCA algorithm
6.3 Explain the relationship between PCA and SVD
Learning Objectives
7.1: Describe the big-picture view of how neural networks work.
7.2: Identify the basic building blocks and notations of deep neural networks.
7.3: Explain how in principle learning is achieved in a deep network.
7.4: Explain key techniques that enable efficient learning in deep networks.
7.5: Appraise the detailed architecture of a basic convolutional neural network.
7.6: Compare the basic concepts and corresponding architecture for recurrent neural networks
and autoencoders.
Learning Outcomes
8.1: Appraise image classification for deep learning
8.2: Appraise video-based inference for deep learning
8.3: Appraise Generative Adversarial Networks (GANs) for deep learning
8.4: Design a deep network using an exemplar application to solve a specific problem
About ASU
Established in Tempe in 1885, Arizona State University (ASU) has developed a new model
for the American Research University, creating an institution that is committed to access,
excellence and impact.
As the prototype for a New American University, ASU pursues research that contributes to the
public good, and ASU assumes major responsibility for the economic, social and cultural vitality
of the communities that surround it. Recognizing the university’s groundbreaking initiatives,
partnerships, programs and research, U.S. News and World Report has named ASU as the most
innovative university all three years it has had the category.
The innovation ranking is due at least in part to a more than 80 percent improvement in ASU’s
graduation rate in the past 15 years, the fact that ASU is the fastest-growing research university
in the country and the emphasis on inclusion and student success that has led to more than 50
percent of the school’s in-state freshman coming from minority backgrounds.
Structured around grand challenges and improving the quality of life on a global scale, the Ira
A. Fulton Schools of Engineering at Arizona State University integrates traditionally separate
disciplines and supports collaborative research in the multidisciplinary areas of biological and
health systems; sustainable engineering and the built environment; matter, transport and energy;
and computing and decision systems. As the largest engineering program in the United States,
students can pursue their educational and career goals through 25 undergraduate degrees or 39
graduate programs and rich experiential education offerings. The Fulton Schools are dedicated
to engineering programs that combine a strong core foundation with top faculty and a reputation
for graduating students who are aggressively recruited by top companies or become superior
candidates for graduate studies in medicine, law, engineering and science.
Creators
Jingrui He is an assistant professor in the School of Computing, Informatics, and Decision
Systems Engineering at Arizona State University. She received her Ph.D. from Carnegie
Mellon University. She joined ASU in 2014 and directs the Statistical Learning Lab (STAR
Lab). Her research focuses on rare category analysis, heterogeneous machine learning,
active learning and semi-supervised learning, with applications in social media analysis,
healthcare, manufacturing process, etc.
Baoxin Li is currently a professor and the chair of the Computer Science & Engineering
Program and a Graduate Faculty Endorsed to Chair in the Electrical Engineering and
Computer Engineering programs. From 2000 to 2004, he was a Senior Researcher with
SHARP Laboratories of America, where he was the technical lead in developing SHARP’s
HiIMPACT Sports™ technologies. He was also an Adjunct Professor with the Portland State
University from 2003 to 2004. His general research interests are on visual computing and
machine learning, especially their application in the context of human-centered computing.