# Non-parametric Approximate Dynamic Programming via the Kernel Method

TL;DR: A novel non-parametric approximate dynamic programming (ADP) algorithm that enjoys graceful approximation and sample complexity guarantees and can serve as a viable alternative to state-of-the-art parametric ADP algorithms.

Abstract: This paper presents a novel non-parametric approximate dynamic programming (ADP) algorithm that enjoys graceful approximation and sample complexity guarantees. In particular, we establish both theoretically and computationally that our proposal can serve as a viable alternative to state-of-the-art parametric ADP algorithms, freeing the designer from carefully specifying an approximation architecture. We accomplish this by developing a kernel-based mathematical program for ADP. Via a computational study on a controlled queueing network, we show that our procedure is competitive with parametric ADP approaches.

...read more

##### Citations

43 citations

30 citations

### Cites background from "Non-parametric Approximate Dynamic ..."

...Following a slightly different line of work, Bhat et al. (2012) propose to kernelize the linear programming formulation of dynamic programming....

[...]

26 citations

17 citations

### Cites methods from "Non-parametric Approximate Dynamic ..."

...…Farias and Van Roy, 2000; Tsitsiklis and Roy, 1996; Tsitsiklis and Van Roy, 1999; Geramifard et al., 2013), approximate linear programming (De Farias and Van Roy, 2003; De Farias and Van Roy, 2004; Desai et al., 2012a), and nonparametric methods are used (Ormoneit and Sen, 2002; Bhat et al., 2012)....

[...]

16 citations

##### References

10,491 citations

7,786 citations

### "Non-parametric Approximate Dynamic ..." refers background in this paper

...For certain sets S, Mercer’s theorem provides another important construction of such a Hilbert space. more examples can be found in the text of Scholkopf and Smola (2001)....

[...]

...The Gaussian kernel is known to be full-dimensional (see, e.g., Theorem 2.18, Scholkopf and Smola, 2001), so that employing such a kernel in our setting would correspond to working with an infinite dimensional approximation architecture....

[...]

...more examples can be found in the text of Scholkopf and Smola (2001)....

[...]

5,541 citations

2,892 citations

### "Non-parametric Approximate Dynamic ..." refers methods in this paper

...Max-Weight (Tassiulas and Ephremides, 1992)....

[...]

...We prepare the ground for the proof by developing appropriate uniform concentration guarantees for appropriate function classes....

[...]

2,201 citations