Holdout cases, also known as validation cases or validation respondents, are a subset of respondents in a conjoint analysis who are not used in the estimation process but are instead held back for the purpose of validating the accuracy of the model. The purpose of using holdout cases is to ensure that the model developed from the analysis is able to accurately predict the preferences of new customers.
The number of holdout cases required in a conjoint analysis depends on various factors such as the sample size, the complexity of the analysis, and the research objectives.
As a general rule of thumb, the number of holdout cases should be at least 10% of the sample size, but it can be increased if the research requires more rigorous validation.
Some criteria that can be used to determine the number of holdout cases required in a conjoint analysis include:
Statistical significance: The number of holdout cases should be large enough to provide statistically significant results. The sample size and the complexity of the analysis can affect the statistical significance of the results.
Research objectives: The number of holdout cases should be based on the research objectives. If the research requires a high level of accuracy, then more holdout cases may be needed.
Data quality: The number of holdout cases should be based on the quality of the data. If the data is of high quality, then fewer holdout cases may be needed.
Model complexity: The number of holdout cases should be based on the complexity of the model. If the model is complex, then more holdout cases may be needed.
In summary, the number of holdout cases required in a conjoint analysis depends on several factors and should be determined based on the research objectives, data quality, statistical significance, and model complexity. A general guideline is to have at least 10% of the sample size as holdout cases.
Source: CHATGPT