ETHZ

Machine Perception course at ETH Zurich

The advances in machine learning, especially in deep learning in recent years, have enabled us to develop systems that can understand several types of data. In this course, we will not only learn what novel approaches are available to perform perceptual tasks such as image understanding, but we will also lay the foundations from basic neural networks to state-of-the-art generative models.

The course consists of normal lectures, tutorials, theory exercises, a semester project and a final written exam.
A general explanation of each part and what other courses I have attended can be found here.

The course is really interesting, many topics and specific models are discussed after a solid mathematical foundation is laid in the beginning. Starting with backpropagation in traditional feedforward networks.
This course contains almost no proofs, however, it requires you to apply some math, like computing partial derivatives with the chain rule and performing backpropagation in more complicated neural networks, like CNN’s or RNN’s.

Grading

The semester project counts 40 % towards the final grade and the remaining 60 % come from a written exam.

Topics

  • Deep Learning introduction
  • How to train neural networks via Backpropagation
  • Convolutional Neural Networks
  • Recurrent Neural Networks
  • Advanced CNN’s
    • Fully Convolutional Neural Networks
  • Generative Models
    • Variational Auto Encoders (VAE)
    • Generative Adversarial Networks (GAN)
  • Autoregressive Models
  • Reinforcement Learning
  • Recent Research

Project

In the summer semester 2020, when I take this course, you can choose from four projects:

  • Dynamic Gesture Recognition
  • 3D Human Pose Estimation
  • Eye Gaze Estimation
  • Human Motion Prediction

My team and I decided to do the Eye Gaze Estimation project.

The task here is to perform eye gaze estimation using pre-processed real-world data extracted from two well-known datasets in this domain (MPIIFaceGaze and GazeCapture).

The problem of eye gaze estimation is to predict the gaze direction in radians (Euler angle representation) based on different features. The features provided are face image, eye-region image, left-eye image, right-eye image, head pose, and face-landmarks.

The project starts in about the middle of the semester and ends a bit after the semester.

It’s graded based on a competitive part among the student-teams and a non-competitive part, which grades the idea/novelty of the approach based on a written report which can be handed in after the project deadline.

For the competitive part, some baselines are given to get a guaranteed grade.