|Bootstrap Aggregation for Cross-Validation and Inference under Constraints
| Dr. Peter Hall
Monday, April 9, 2007
A major asset of the cross-validation approach to smoothing-parameter choice is its utilitarian character. However, the bandwidths produced by cross-valdiation are relatively noisy, and this difficulty impedes good performance. The stochastic variability of cross-validation can be reduced significantly by using bootstrap aggregation, or bagging, a method proposed by Breiman (1999). The technique is very simple to use, and enjoys the utilitarian character of cross-validation. For instance, it can be applied in practically all of the many settings where cross-validation is employed for bandwidth choice with the aim of optimising an $L_2$ measure of performance.
Arbitrarily large reductions in bandwidth variability are theoretically possible, although in practice bagging would likely be used relatively modestly. In particular, half-sample bagging can reduce bandwidth variability by approximately 50%. We shall also discuss a bagging-based approach to constrained inference, for example to parameter estimation when it is known that the true value of the parameter satisfies an inequality constraint.