Currently, I teach following courses:
Introduction to Reinforcement Learning for Optimal Control
Lecture Slides:
Computer Science-AI Centric (2024-2025)
Session 1: Introduction, Motivation, Markov Decision Process
Session 2: MDP, V-functions, Q-functions, Bellman Equations and Notion of Optimality
Session 3: Dynammic Programming, Policy iteration, Value Iteration.
Session 4: Model Free prediction and Control (SARSA, Q-learning)
Session 5: Function approximation (Deep Neural Networks as Approximators), Deep Q Networks
Session 6 Policy gradient approaches, Deep deterministic Policy gradient (DDPG)
Tutorials: TD0(SARSA, Qleanring Cartpole and Moutain Car) and TD1 (Introduction to Open AI Gym Env, TD control: SARSA, Q-learning),
TD2: DQN on Lunar Lander
Home Assignment (DM1)
TD3: DDPG Hands on ( Bipedal Walker, Inverted_Pendulum, Robot(Pusher) )
Student Feedback 2024-25
Control Theory Centric ()
Session 1:
Introduction to Reinforcement Learning,
Markov Decision Process,
Backward recursive Relation
Session 2
Dynamic Programming,
Bellman Equation and Bellman Optimality Equation
Discrete time Linear system control
Session 3
Policy Iteration
Policy Iteration Algorithm
Discrete time Linear Quadratic Regulator
Session 4
Temporal Difference
Function Approximation
Reinforcement Learning for DT Nonlinear system
Session 5
Q-functions, Policy Iteration using Q functions,
Q-learning
Session 6
Policy gradient approaches,
Deep deterministic Policy gradient (DDPG)
Tutorials (TD):
TD1 : Policy Iteration based learning Feedback optimal control law for Linear system in Discrete Time
TD2 : Policy Iteration for unknown Linear Systems using neural networks
TD3 : Hands on Deep Q-N and DDPG using MATLAB
Deep Reinforcement Learning (DQN and DDPG).
- Environments solved in Tutorials (OpenAI gym based): Cartpole, Inverted Pendulum and Bipedal Walker
Sûreté de fonctionnement et Retour d'expériences ( Reliability and Feedback Data Collection) ,
4th Year of Engineering in Management opérationnel Maintenance et Maîtrise des risques (M3)
- Course Slides: CM1, CM2, CM3, CM4, CM4b, CM5etCM6
- TDs (Tutorials): TD1, TD2, TD3, TD4
- Solutions TD: TD1, TD2, TD3, TD3(Exo4 et 5). , TD4
- Gamma Function Table
Introduction to Deep Learning and Artificial Intelligence (5th year, Ingénierie de l’Information et des Systèmes (FISE & FISA) :
- Lectures:
- Practicals (Jupyter notebooks, FISE ONLY):
- ANN Basics & classification with MNIST data (TD1, House_Price, TD2,)
- CNN based: Construct LeNET styled CNN for MINIST data classification (TD_LeNet) TD_CIFAR_COVNET
- LSTM based: prediction of time series (TD5)
- Student Feedback ( in French) 2019-20, 2020-2021 , FISA 2023-24
Artificial Intelligence for Prognostics (5th Year, Department M3)
Neural Networks for decision making
CNNs basics and CNNs for Prognostics
RNNs, LSTMs Basics and Deep LSTMs for Prognostics
Course Slides,
Tutorials: TD1, TD1a, TD1b (Battery Degradation Prognostics) ,
CMAPSS Introduction and LSTM for RUL Prediction
Student Feedback: 2023-2024
Control of Mobile Robots (5A IA2R Parcours SIA)
Lab 1: The QCAR (Autonomous Car),
Lab 2: The QBOT (differential drive robot by Quanser)
Lab3: The 3Pi+ Robot
Lab4: The Zumo robot