Reproducing Kernel Hilbert Spaces In Probability And Statistics

The reproducing kernel Hilbert space construction is a bijection or transform theory which associates a positive definite kernel (gaussian processes) with a Hilbert space offunctions. Like all transform theories (think Fourier), problems in one space may become transparent in the other, and optimal solutions in one space are often usefully optimal in the other. The theory was born in complex function theory, abstracted and then accidently injected into Statistics; Manny Parzen as a graduate student at Berkeley was given a strip of paper containing his qualifying exam problem- It read "reproducing kernel Hilbert space"- In the 1950's this was a truly obscure topic. Parzen tracked it down and internalized the subject. Soon after, he applied it to problems with the following fla­ vor: consider estimating the mean functions of a gaussian process. The mean functions which cannot be distinguished with probability one are precisely the functions in the Hilbert space associated to the covariance kernel of the processes. Parzen's own lively account of his work on re­ producing kernels is charmingly told in his interview with H. Joseph Newton in Statistical Science, 17, 2002, p. 364-366. Parzen moved to Stanford and his infectious enthusiasm caught Jerry Sacks, Don Ylvisaker and Grace Wahba among others. Sacks and Ylvis­ aker applied the ideas to design problems such as the following. Sup­ pose (XdO