2015 QPRC

Model Selection Strategies for Definitive Screening Designs

Maria L. Weese, Ph.D.      Douglas Montgomery, Ph.D.           Philip J. Ramsey, Ph.D.
University of Miami, OH    Arizona State University       University of New Hampshire              

Abstract

Jones and Nachtscheim (2011) introduced a new type of highly efficient experimental design entitled Definitive Screening Designs (DSDs).  The designs are unique in that all factors in the experiment have three levels (allowing for the potential estimation of quadratic effects), main effects are orthogonal and free of aliasing, and no two factor interaction or quadratic effect is fully confounded with another effect; all of this in as little as 2K+1 trials for K factors.  Although the DSDs are starting to be adopted in industry, especially in biotechnology, they do present the experimenters with unique opportunities and challenges in terms of analysis. 

In this talk we will discuss the results of a comprehensive simulation study to evaluate several possible approaches to the analysis of the results from DSD experiments. The simulation scenarios vary the number of experimental factors and the number of active effects: main, quadratic, and two-way interaction. The simulation focuses on three possible model selection strategies (AICc or BIC is used as a selection criterion):

1. Forward Selection
2. Dantzig Selector
3. All Subsets Regression

The techniques will be contrasted and compared in terms of the Power (defined as the ability to correctly identify the true active factors) to detect active effects as well as the number of times inactive effects are identified as active (Type I errors) and predictive ability.

Additionally, a case study will be discussed where a DSD was run in parallel with a CCD. The discussion will focus on how well the two designs agree in terms of active effects and provide recommendations for analysis of these designs.