Robert Babuska, Lucian Busoniu, and Bart de Schutter are with the Delft University of Technology. Damien Ernst is with the University of Liege.

Introduction. Dynamic programming and reinforcement learning. Focus of this book. Book outline. Basics of dynamic programming and reinforcement learning. Introduction. Markov decision processes. Value iteration. Policy iteration. Direct policy search. Conclusions. Bibliographical notes. Dynamic programming and reinforcement learning in large and continuous spaces. Introduction. The need for approximation in large and continuous spaces. Approximate value iteration. Approximate policy iteration. Finding value function approximators automatically. Approximate policy search. Comparison of approximate value iteration, policy iteration, and policy search. Conclusions. Bibliographical notes. Q-value iteration with fuzzy approximation. Introduction. Fuzzy Q-iteration. Analysis of fuzzy Q-iteration. Optimizing the membership functions. Experimental studies. Conclusions. Bibliographical notes. Online and continuous-action least-squares policy iteration. Introduction. Least-squares policy iteration. LSPI with continuous-action approximation. Online LSPI. Using prior knowledge in online LSPI. Experimental studies. Conclusions. Bibliographical notes. Direct policy search with adaptive basis functions. Introduction. Policy search with adaptive basis functions. Experimental studies. Conclusions. Bibliographical notes. References. Glossary.