# diochnos/teaching/CS4033-5033/2022S

## CS 4033/5033 – Machine Learning (Spring 2022)

The class is cross-listed as CS 4033 and CS 5033, so that both undergraduate and graduate students can enroll simultaneously. No student may earn credit for both 4033 and 5033.

### Table of Contents

### Course Description

Topics include decision trees, relational learning, neural networks, Bayesian learning, reinforcement learning, multiple-instance learning, feature selection, learning appropriate representations, clustering, and kernel methods. No student may earn credit for both 4033 and 5033.

[Course Description] [Table of Contents] [Top]

### Basic Information

#### Syllabus

The syllabus is available here.

#### Time and Location

Mondays, Wednesdays, and Fridays, 11:30am – 12:20pm, Devon Energy Hall 120.

#### Contact Information

Please see here.

#### Teaching Assistants

The teaching assistant for the class is Taras Basiuk.

#### Office Hours

We will be holding our office hours at the following times.

- Mondays
- 2:00pm – 3:00pm, 244 Devon Energy Hall & Online (Dimitris)
- Tuesdays
- 10:00am – 11:00am, Online (Taras)
- Wednesdays
- 2:00pm – 3:00pm, 244 Devon Energy Hall & Online (Dimitris)
- Thursdays
- 10:00am – 11:00am, Online (Taras)

Zoom information about office hours is available on the Syllabus page on Canvas.

##### Exceptions to the Regular Schedule of Office Hours

If you want to meet me outside of my office hours, please send me an email and arrange an appointment.

##### Exceptions to the Regular Schedule of Office Hours for the TAs

As exceptions appear along the way, they will also be announced here.

[Basic Information] [Table of Contents] [Top]

### Important Coronavirus-Related Information

We have the following links.

[Important Coronavirus-Related Information] [Table of Contents] [Top]

### Homework Assignments

Assignment 1: Announce on Mon, Jan 24, 2022. Due on Wed, Feb 2, 2022.

Assignment 2: Announce on Mon, Feb 7, 2022. Due on Mon, Feb 21, 2022.

Assignment 3: Announce on Mon, Feb 28, 2022. Due on Fri, Mar 11, 2022.

Assignment 4: Announce on Fri, Mar 11, 2022. Due on Wed, Mar 30, 2022.

Assignment 5: Announce on Wed, Mar 30, 2022. Due on Mon, Apr 11, 2022.

Assignment 6: Announce on Mon, Apr 11, 2022. Due on Fri, Apr 29, 2022.

[Homework Assignments] [Table of Contents] [Top]

### Projects

Information related to the projects will show up here.

#### Ideas for projects

Below are some ideas for your projects.

##### Reinforcement Learning Ideas

- OpenAI Gym:
provides an interface for training your own RL agent to play a computer game.

(Try to select a simple game so that it is easier to deal with it.) - Simple board games for RL.
- Variations of bandit problems.

##### Supervised Learning Ideas

- Kaggle: has lots of datasets and you may actually be able to participate in a competition and see how your algorithm and your approach compares against others.
- KDnuggets: datasets from KD nuggets.
- UCI repository: datasets that are available from the repository that is maintained by the University of California, Irvine.
- CIFAR-10 and CIFAR-100
- ImageNet
- SVHN (Street-View House Numbers)

[Projects] [Table of Contents] [Top]

### Milestones

Week 2: Homework 1 is announced (beginning of week).

Week 3: Homework 1 is due (mid-week). In-class presentations for the reinforcement learning project (end of week).

Week 4: Homework 2 is announced (beginning of week). Project written proposal is due (beginning of week).

Week 6: Homework 2 is due (beginning of week). Project checkpoint is due (end of week).

Week 7: Homework 3 is announced (beginning of week).

Week 8: Homework 3 is due and homework 4 is announced (end of week).

Week 9 (**Spring Break**): Reinforcement learning project is due at the end
of week.

Week 10: In-class presentations for the supervised learning project (end of week).

Week 11: Homework 4 is due and homework 5 is announced (mid-week). Project written proposal is due (beginning of week).

Week 13: Homework 5 is due and homework 6 is announced (beginning of week). Project checkpoint is due (end of week).

Week 15: Homework 6 is due (end of week).

Week 16: Supervised learning project is due (end of week).

[Milestones] [Table of Contents] [Top]

### Machine Learning Resources

#### Books

The two books that we plan to use for the course are available for free in electronic format in the following links:

- Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto. (1st edition)
- The Elements of Statistical Learning, by Trevor Hastie, Robert Tibshirani, and Jerome Friedman.

Another book that I like a lot and recommend to people who are starting with machine learning is the following one:

- Machine Learning, by Tom Mitchell.

#### Personal Notes

#### Notes by Others

- Recall, Precision, F1, ROC, AUC, and everything, by Ofir Shalev. (local pdf copy)
- Neural Networks and Deep Learning, by Michael Nielsen.

#### Papers

- Machine Learning that Matters, by Kiri L. Wagstaff
- A Few Useful Things to Know About Machine Learning, by Pedro Domingos. (alternate version)
- Perceptron-based learning algorithms, by Stephen I. Gallant. (Optional reading; this is the paper for the pocket algorithm.)

[Machine Learning Resources] [Table of Contents] [Top]

### Class Log

A log for the class will be held online here.

#### Week 1

#### Class 1 (Jan 19, 2022)

About this Course.

Discussion on syllabus and policies.

#### Class 2 (Jan 21, 2022)

Discussion on projects. Introduction to Machine Learning.

Pretest in class.

Assigned Reading: Elements of Statistical Learning (ESL), Chapter 1.

Assigned Reading: Sutton & Barto: Chapters 1 and 2.

Assigned today: Think about short and long projects. Think about the topic for your RL project.

#### Week 2

#### Class 3 (Jan 24, 2022)

Assigned today: Homework 1.

Introduction to reinforcement learning.

Basic ingredients of RL methods: policy, value function, model.

#### Class 4 (Jan 26, 2022)

Continued our discussion on introduction to reinforcement learning.

The prediction problem and the control problem.

Markov Decision Processes (MDPs).

Discussion on the Bellman Expectation Equations. Backup diagrams and solution of the prediction problem using linear algebra. Revisited the recycling robot example and we showed how we can evaluate the policy that picks an action with the same probability at each of the two energy states of the robot.

To the extent possible we will discuss the control problem.

#### Class 5 (Jan 28, 2022)

TBA

#### Week 3

#### Class 6 (Jan 31, 2022)

TBA

#### Class 7 (Feb 2, 2022)

Due today: Homework 1.

Proposals for reinforcement learning projects; in-class as well as remote presentations.

#### Class 8 (Feb 4, 2022)

Proposals for reinforcement learning projects; in-class as well as remote presentations.

#### Week 4

#### Class 9 (Feb 7, 2022)

Due today: Written proposal for the reinforcement learning project; whether short or long.

TBA

#### Class 10 (Feb 9, 2022)

Assigned today: Homework 2.

TBA

#### Class 11 (Feb 11, 2022)

TBA

#### Week 5

#### Class 12 (Feb 14, 2022)

TBA

#### Class 13 (Feb 16, 2022)

TBA

#### Class 14 (Feb 18, 2022)

TBA

#### Week 6

#### Class 15 (Feb 21, 2022)

Due today: Homework 2.

TBA

#### Class 16 (Feb 23, 2022)

Assigned today: Homework 3.

TBA

#### Class 17 (Feb 25, 2022)

Due today: Reinforcement learning checkpoint.

TBA

#### Week 7

#### Class 18 (Feb 28, 2022)

TBA

#### Class 19 (Mar 2, 2022)

TBA

#### Class 20 (Mar 4, 2022)

TBA

#### Week 8

#### Class 21 (Mar 7, 2022)

TBA

#### Class 22 (Mar 9, 2022)

TBA

#### Class 23 (Mar 11, 2022)

TBA

#### Week 9

#### Mar 14, 2022

Spring break; no classes.

#### Mar 16, 2022

Spring break; no classes.

#### Mar 18, 2022

Due today: Reinforcement learning project.

Spring break; no classes.

#### Week 10

#### Class 24 (Mar 21, 2022)

TBA

#### Class 25 (Mar 23, 2022)

TBA

#### Class 26 (Mar 25, 2022)

Due today: Reinforcement learning project.

TBA

#### Week 11

#### Class 27 (Mar 28, 2022)

TBA

#### Class 28 (Mar 30, 2022)

TBA

#### Class 29 (Apr 1, 2022)

TBA

#### Week 12

#### Class 30 (Apr 4, 2022)

TBA

#### Class 31 (Apr 6, 2022)

TBA

#### Class 32 (Apr 8, 2022)

TBA

#### Week 13

#### Class 33 (Apr 11, 2022)

TBA

#### Class 34 (Apr 13, 2022)

TBA

#### Class 35 (Apr 15, 2022)

TBA

#### Week 14

#### Class 36 (Apr 18, 2022)

TBA

#### Class 37 (Apr 20, 2022)

TBA

#### Class 38 (Apr 22, 2022)

TBA

#### Week 15

#### Class 39 (Apr 25, 2022)

TBA

#### Class 40 (Apr 27, 2022)

TBA

#### Class 41 (Apr 29, 2022)

TBA

#### Week 16

#### Class 42 (May 2, 2022)

TBA

#### Class 43 (May 4, 2022)

TBA

#### Class 44 (May 6, 2022)

TBA

#### Wednesday, May 11, 2022 (1:30pm – 3:30pm)

Due today: Supervised learning project (whether short or long) write-up and source code.

Normally this would be the date and time of the final exam. However, we will not have a final exam as the class has a semester-long project.

[Class Log] [Table of Contents] [Top]