Someone wanted to learn this too, so Grasp built them a personal learning path.
You can keep exploring this one or create a path like this one shaped around you.
Recommended resources
Module 2
Modeling and Simulating Finite MDPs in Python
Module 3
Finite-Horizon Evaluation and Control
Module 4
Infinite-Horizon Discounted Policy Evaluation
Module 5
Bellman Optimality and Value Iteration
Module 6
Policy Improvement and Policy Iteration
Module 7
Linear Programming and Occupancy Measures
Module 8
Stochastic Shortest Paths and Average-Reward MDPs
Module 9
Sensitivity, Reward Design, and Robust Planning
Module 10
Scalable and Approximate Planning
Module 11
Bridge from Known Models to Reinforcement Learning
Module 12