diochnos/teaching/CS4033-5033/2025S
CS 4033/5033 – Machine Learning Fundamentals (Spring 2025)
The class is cross-listed as CS 4033 and CS 5033, so that both undergraduate and graduate students can enroll simultaneously. No student may earn credit for both 4033 and 5033.
Table of Contents
Course Description
Topics include decision trees, relational learning, neural networks, Bayesian learning, reinforcement learning, multiple-instance learning, feature selection, learning appropriate representations, clustering, and kernel methods. No student may earn credit for both 4033 and 5033.
[Course Description] [Table of Contents] [Top]
Basic Information
Syllabus
The syllabus is available here.
Time and Location
Mondays, Wednesdays, and Fridays, 11:00am – 11:50am, Devon Energy Hall, Room 120.
Contact Information
Please see here.
Teaching Assistants
Tommy Pham (Tommy.T.Pham-1) is a teaching assistant for this course. If you want to send an email to your TA, please use the prefix that I have inside the parentheses, followed by the @ sign, followed by the usual way all email addresses use a domain for emails at OU.
Office Hours
We will be holding our office hours at the following times.
- Mondays
- 2:00pm – 2:50pm, 230 Devon Energy Hall (Dimitris)
- Tuesdays
- 12:00pm – 1:00pm, 115 Devon Energy Hall (Tommy)
- Wednesdays
- 3:00pm – 3:50pm, 230 Devon Energy Hall (Dimitris)
- Friday
- 1:00pm – 2:00pm, 115 Devon Energy Hall (Tommy)
Exceptions to the Regular Schedule of Office Hours
If you want to meet me outside of my office hours, please send me an email and arrange an appointment.
Wednesday, March 12: I will be holding office hours between 3:30pm – 3:50pm. That is, only the second half of the hour I will be available to see students as I have a conflict with another appointment during the first half of the hour.
Monday, April 7: I will be holding office hours between 2:30pm – 2:50pm. That is, only the second half of the hour I will be available to see students as I have a conflict with an MS Exit Exam during the first half of the hour.
[Basic Information] [Table of Contents] [Top]
Homework Assignments
Assignment 1: Announce on Wed, Jan 22, 2025. Due on Sun, Feb 2, 2025.
Assignment 2: Announce on Fri, Jan 31, 2025. Due on Sat, Feb 15, 2025.
Assignment 3: Announce on Mon, Feb 17, 2025. Due on Mon, Mar 3, 2025.
Assignment 4: Announce on Sat, Mar 1, 2025. Due on Wed, Mar 19, 2025.
Assignment 5: Announce on Wed, Mar 19, 2025. Due on Wed, Apr 2, 2025.
Assignment 6: Announce on Wed, Apr 2, 2025. Due on Sat, Apr 26, 2025.
[Homework Assignments] [Table of Contents] [Top]
Projects
Information related to the projects will show up here.
Ideas for projects
Below are some ideas for your projects.
Reinforcement Learning Ideas
- Gymnasium from Farama Foundation (continuation of the OpenAI Gym):
provides an interface for training your own RL agent to play a computer game.
(Try to select a simple game so that it is easier to deal with it.) - Simple board games for RL.
- Variations of bandit problems.
- A gym environment for the classic game of snake is available here.
- Avoid constraint satisfaction problems (e.g., Sudoku, Worldle, etc.)
Supervised Learning Ideas
Before we discuss any ideas, as a reminder, you cannot use MNIST, because that dataset has been studied with any conceivable algorithm at this point and all the information is available for free online.
- UCI repository: datasets that are available from the repository that is maintained by the University of California, Irvine.
- Make Moons: a synthetic dataset where you can also compare the performance of the models that you will develop from scratch with the equivalent ones from scikit-learn.
- Kaggle: has lots of datasets and you may actually be able to participate in a competition and see how your algorithm and your approach compares against others.
- KDnuggets: datasets from KD nuggets.
- Smiling or not?
- Fashion MNIST
- ImageNet (Though, I would suggest you work with the Tiny ImageNet)
- CIFAR-10 and CIFAR-100
- SVHN (Street-View House Numbers)
- Please note that ImageNet, CIFAR-10, CIFAR-100, and SVHN, are all difficult datasets.
- As a sidenote, if you want to reduce the dimensionality of some datasets (so that you can speed up computations), you can use dimension reduction techniques (e.g., Principal Component Analysis). You are free to use libraries exclusively for this task.
[Projects] [Table of Contents] [Top]
Milestones
Week 2: Homework 1 is announced (beginning of week).
Week 3: Homework 1 is due (mid-week). Homework 2 is announced (end of week). In-class presentations for the reinforcement learning project (end of week).
Week 4: Homework 2 is announced (beginning of week). Project written proposal is due (beginning of week).
Week 5: Homework 2 is due (end of week).
Week 6: Homework 3 is announced (beginning of week). Review discussion for first midterm (end of week).
Week 7: Midterm 1 (beginning of week). Project checkpoint is due (mid-week). Homework 4 is announced (end of week).
Week 8: Homework 3 is due (beginning of week).
Week 9: Proposals for supervised learning project in class (mid-week).
Week 10 (Spring Break): Supervised learning project written proposals due (beginning of week). Homework 4 is due and homework 5 is announced (mid-week). Reinforcement learning project is due (end of week).
Week 12: Homework 5 is due and homework 6 is annouced (mid-week). Review discussion for second midterm (end of week).
Week 13: Midterm 2 (beginning of week). Supervised learning project checkpoint is due (end of week; Saturday).
Week 15: Homework 6 is due (end of week).
Week 16: Ask me anything (end of week). Supervised learning project is due (end of week). It should be ok if you submit your work early during the finals week (i.e., at least 3 days before the final exam).
Week 17: Final exam.
[Milestones] [Table of Contents] [Top]
Machine Learning Resources
Books
The three books that we plan to use for the course are available for free in electronic format in the following links:
- Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto. (1st edition)
- An Introduction to Statistical Learning (with Applications in Python), by Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.
- Machine Learning, by Tom Mitchell.
Personal Notes
Notes by Others
- Recall, Precision, F1, ROC, AUC, and everything, by Ofir Shalev. (local pdf copy)
- Neural Networks and Deep Learning, by Michael Nielsen.
- Animation comparing ridge and lasso regression.
Papers
- Machine Learning that Matters, by Kiri L. Wagstaff.
- A Few Useful Things to Know About Machine Learning, by Pedro Domingos. (alternate version)
- Machine learning: Trends, perspectives, and prospects, by Michael Jordan and Tom Mitchell.
- Perceptron-based learning
algorithms, by Stephen I. Gallant.
- Optional reading; this is the paper for the pocket algorithm. Regarding the implementation, you may find the algorithm listed in the paper Perceptrons Under Verifiable Random Data Corruption easier to code. This last paper was written by an undergraduate student at OU, Jose Aguilar Escamilla, and myself.
[Machine Learning Resources] [Table of Contents] [Top]
Class Log
A log for the class will be held online here.
Week 1
Class 1 (Jan 13, 2025)
About this Course.
Discussion on syllabus and policies.
Class 2 (Jan 15, 2025)
Discussion on projects. Introduction to Machine Learning.
Assigned Reading: Elements of Statistical Learning (ESL), Chapter 1.
Assigned Reading: Sutton & Barto: Chapters 1 and 2.
Assigned today: Start thinking about the topic for your RL project.
Class 3 (Jan 17, 2025)
Introduction to reinforcement learning.
Basic ingredients of RL methods: policy, value function, model.
Discussion on the projects, deadlines, and various expectations. Also, where we can find certain information on Canvas.
Week 2
Monday, January 20, 2025
No class today;Martin Luther King day.
Classes 4-5 (Jan 22-24, 2025)
Assigned (Mon): Homework 1.
Exploration vs Exploitation. The multi-armed bandit problem from the book (Chapter 2).
The prediction problem and the control problem.
Markov Decision Processes (MDPs).
Discussion on the Bellman Expectation Equations. Backup diagrams and solution of the prediction problem using linear algebra. Revisited the recycling robot example and we showed how we can evaluate the policy that picks an action with the same probability at each of the two energy states of the robot.
Bellman optimality equations and the control problem.
Assigned Reading (Mon, 1/30/2023): Sutton & Barto: Chapter 3.
Week 3
Classes 6-7-8 (Jan 27-29-31)
Dynamic programming methods. Policy iteration, value iteration, as well as we had some last remarks on dynamic programming (complexity, asynchronous backups, etc.)
Started our discussion on model-free methods that are used for prediction.
Fri, 1/31/2025: Proposals for reinforcement learning projects; in-class presentations.
Due (Feb 2): Homework 1.
Week 4
Classes 9-10-11 (Feb 3-5-7)
Assigned (Mon, Feb 3): Homework 2.
Due Mon, Feb 3: Written proposal for the reinforcement learning project.
Assigned Reading: Sutton & Barto: Chapter 4.
Overview of Monte-Carlo and Temporal Difference learning.
First-visit and every-visit Monte Carlo methods. Application to Blackjack.
Iterative calculation of empirical average.
Temporal difference learning. Application to the "Return Home" example.
Comparison of Monte Carlo and Temporal Difference learning.
Assigned Reading (Wed): Sutton & Barto: Sections 5.1, 5.2, 6.1, 6.2, 6.3.
n-Step Returns and Eligibility Traces. Forward view, backward view, and equivalence.
Assigned Reading: Sutton & Barto: Sections 7.1, 7.2, 12.1, 12.2.
The control problem. Using $\varepsilon$-greedy policy in order to guarantee enough exploration of the state space so that we are able to calculate accurate optimal values for the value functions.
Solution with an on-policy Monte-Carlo approach.
Assigned Reading (Mon): Sutton & Barto: Sections 5.3, 5.4.
Week 5
Classes 12-13-14 (Feb 10-12-14)
Discussion on information about the class and the second homework.
Continue our discussion on solving the control problem. This time we use the idea of TD learning which leads to Sarsa and we will also discuss the extension to Sarsa($\lambda$).
Assigned Reading (Wed): Sutton & Barto: Sections 6.4, 6.5, 12.7.
Solving the control problem using an off-policy method: Q-Learning.
Introduction to function approximation.
How can we do linear function approximation and solve the prediction and the control problem using our basic methods.
Simple ways to construct features: state aggregation, coarse coding, tile coding. The book has more examples.
Discussion of some examples with function approximation. Among them, Sarsa with linear function approximation on the Mountain Car problem.
Assigned Reading: Sutton & Barto: Sections 9.1 – 9.5, 10.1, 10.2.
Due Sat: Homework 2.
Week 6
Classes 15-16 (Feb 17-19)
Assigned today: Homework 3.
Introduction to supervised learning.
What is supervised learning? Regression and classification problems. The role of inductive bias.
Definitions on terminology that we will be using throughout the course for supervised learning.
Assigned Reading: ISL 2.1.
Further discussion on the notion of the hypothesis space $\mathcal{H}$. The different algorithms that we will see on supervised learning, largely define how we will perform the seach in this space and come up with a hypothesis $h$ that we will believe will approximate well the ground truth $c$.
Introduction to nearest neighbor learning. Example with flowers based on sepal length and sepal width.
1-Nearest Neighbor classification and connection to the Voronoi Diagram (this is a topic discussed in Computational Geometry classes).
Assigned Reading: Nearest neighbors: Mitchell 8.1, 8.2 – 8.2.3. Alternatively, ISL 2.2.3 (classification) and 3.5 (regression).
Class 17 (Feb 21)
Review discussion on reinforcement learning for first midterm.
Week 7
Classes 18 (Feb 24)
First midterm.
Classes 19 and 20 (Feb 26-28)
Due on Feb 26: Reinforcement learning checkpoint.
Continue our discussion on Nearest Neighbors. Different metrics. Application to regression problems. Distance-weighted nearest neighbor method.
Naive Bayes for classification. Example on PlayTennis.
The m-estimate and dealing with various corner cases.
Naive Bayes for document classification.
Discussion on creating features for document classification: bag-of-words, n-grams, TF-IDF.
Assigned Reading: Naive Bayes: Mitchell 6.9, 6.10. You can also have a look in ISL Section 4.4.4.
Gaussian Naive Bayes (dealing with continued-valued attributes) and other variants of Naive Bayes for classification.
Naive Bayes is not used for regression problems.
Discussion on risk and empirical risk and how these quantities look like when we use the 0-1 loss $\ell_{\text{0-1}}$ or the square loss $\ell_{\text{sq}}$. Empirical Risk Minimization (ERM) principle.
Some remarks on terminology and notation used in statistics and in computer science.
Assigned Reading: Please pay attention to the slides for the discussion on loss functions, risk, empirical risk, the ERM principle, etc.
Assigned Reading: You can also look into ISL Section 2.2.1.
Saturday, March 1
Assigned today: Homework 4.
Week 8
Class 21 (Mar 3)
Due today: Homework 3.
Discussion of solutions for midterm 1.
Classes 22-23 (Mar 5-7)
Started our discussion on linear models. Perceptrons, decision boundary, discussion on the representational power of various functions. The update rule that is used for learning using perceptrons.
Assigned Reading: Perceptrons: Mitchell 4.1 – 4.4.2.
Perceptrons on linearly separable data and non-linearly separable data. Transformations that allow the perceptron to learn non-linearly separable data and issues of these transformations. The pocket algorithm.
Week 9
Class 24 (Mon, Mar 10)
Linear regression. Ordinary least squares solution.
Solve linear regression problems using gradient descent.
Assigned Reading: Mitchell 4.4.3 – 4.4.4. Alternatively, ISL 3.1 – 3.3 and 3.5.
Class 25 (Wed, Mar 12)
Proposals for supervised learning projects; in-class presentations.
Logistic regression. How is it different from other linear models? Logistic loss.
Class 26 (Fri, Mar 14)
Conclude our discussion on the logistic loss and ultimately on logistic regression. Termination criteria for logistic regression using gradient descent.
Assigned Reading: Logistic regression: ISL 4 – 4.3.5.
Week 10
Mon, Mar 17
Spring break; no class.
Due today: Written proposals due for supervised learning project.
Wed, Mar 19
Spring break; no class.
Due today: Homework 4.
Assigned today: Homework 5.
Fri, Mar 21
Spring break; no class.
Sunday, March 23 (End of spring break)
Due today: Reinforcement learning project write-up and source code.
Week 11
Classes 27-28-29 (Mar 24-26-28)
Introduction to regularization and stability. Structural Risk Minimization (SRM) and Regularized Loss Minimization (RLM).
Regression and regularization. Ridge regression, lasso regression, and elastic-net. Why lasso is more likely to create a sparser model compared to ridge regression.
Discussion on overfitting.
Week 12
Classes 30-31-32 (Mar 31 and Apr 2-4)
Class 30 (Mar 31)
Discussion on assessing the goodness of fit of linear models. Discussed: Residual Standard Error (RSE), R2 statistic, adjusted R2 statistic, residual plots, Cp, AIC, and BIC criteria.
Bias – variance tradeoff. Underfitting and overfitting and the general ideas behind model selection.
Holdout method using 2-way and 3-way partitioning of the given dataset.
Start discussion on cross-validation.
Assigned Reading: A Few Useful Things to Know About Machine Learning , by Pedro Domingos.
Discussion in class on issues that arise in supervised learning algorithms that we have covered so far: perceptron and bound on number of mistakes, stability, PAC guarantees and distributional assumptions.
Conclude our discussion on cross-validation. Leave-One-Out-Cross-Validation (LOOCV). Stratified cross-validation.
Assigned Reading: ISL Sections 2.2.2 and 2.2.3, Chapter 5, Section 6.2.
Due today: Written proposals due for supervised learning project (Section 2).
Class 31 (Apr 2)
Metrics Beyond Low Risk.
A nice decomposition of the instance space and ultimately of the predictions that we make in a dataset, by using false positives, false negatives, true positives, and true negatives - regions in the instance space, or counting examples from the dataset.
The confusion matrix.
Complex performance measures based on: recall, precision, specificity. Balanced accuracy and F1-score.
Receiver Operator Characteristic (ROC) curve.
Assigned Reading: ISL second part of the discussion in Section 4.4.2.
Class 32 (Apr 4)
Artificial neural networks.
Popular activation functions (and their derivatives).
The backpropagation algorithm.
Explanation of the phenomenon of vanishing gradients.
Due today: Homework 5 (Section 1).
Assigned today: Homework 6 (Both Sections).
Sun, Apr 6
Due today: Homework 5 (Section 2).
Week 13
Classes 33-34-35 (Apr 7-9-11)
Classes 33-34 (Apr 7-9)
Derivation of the backpropagation rule.
An illustrative example on artificial neural networks. Design decisions and results.
Additional comments and discussion on artificial neural networks.
Assigned Reading: Mitchell Chapter 4. ISL Sections 10.1 – 10.2, 10.6 – 10.7.
Classes 35 (Apr 11)
Review before the second midterm (mostly on supervised learning).
Sat, Apr 12
Due: Supervised Learning Project Checkpoint (Section 1).
Week 14
Class 36 (Apr 14)
Second midterm.
Classes 37-38 (Apr 16-18)
Decision trees for classification and regression.
Assigned Reading: Mitchell Chapter 3. ISL Section 8.1.
Fri, Apr 18
Due: Supervised Learning Project Checkpoint (Section 2).
Week 15
Classes 39-40-41 (Apr 21-23-25)
Discussion of the solutions for the second midterm.
Ensembles. Bootstrap and bagged predictors. Random forests. Boosting methods.
Assigned Reading: ISL Section 8.2.
Start unsupervised learning.
Sat, Apr 26
Due today: Homework 6.
Week 16
Classes 42-43-44 (Apr 28-30 and May 2)
Unsupervised learning: principal component analysis and clustering. (We may cover a subset of these depending on the pace and other plans; e.g., guest lecture.)
Assigned Reading: ISL Sections 12.1 – 12.2 and 12.4.
Potentially a guest lecture.
Devote some time in class for the student evaluations.
Advertisement of computational learning theory.
Ask me anything.
Due Friday, May 2: Supervised learning project write-up and source code.
Friday, May 9, 2025 (1:30pm – 3:30pm)
Final exam, taking place in the same room that we had our regular meetings throughout the entire semester.
[Class Log] [Table of Contents] [Top]
