Prism Validation

[Marketta Carucci]

Iam trying to write a WPF application using c# and the with the help of Prism 6.3 library. I watched all available tutorials on pluralsight.com for Prism by @BrianLagunas . But not of them show how to do data validation.

If you need complex validation (i.e. no single property value is valid, but only a combination of values of differnet properties), then your view model needs to implement INotifyDataErrorInfo. You basically do your validation in any of the affected properties' setters and finally raise ErrorsChanged...

The Prevention Impacts Simulation Model (PRISM), a system dynamics model that simulates health, mortality, and economic outcomes for the US population, has been used to support community-level strategic planning in several US communities and to evaluate the potential long-term effects of community initiatives to reduce chronic disease and its risk factors.

We applied the model validation framework developed independently by the International Society of Pharmacoeconomics and Outcomes Research and the Society for Medical Decision Making modeling task force to validate PRISM. This framework included model review by external experts and quantitative data comparison by the study team.

External expert review determined that PRISM is based on up-to-date science. One-way sensitivity analysis showed that no parameter affected results by more than 5%. Comparison with other published models, such as ModelHealth, showed that PRISM produces lower estimates of effects and cost savings. Comparison with surveillance data showed that projected model trends in risk factors and outcomes align closely with secular trends. Four measures did not align with surveillance data, and those were recalibrated.

PRISM is a useful tool to simulate the potential effects and costs of public health interventions. Results of this validation should help assure health policy leaders that PRISM can help support community health program planning and evaluation efforts.

Public health approaches to address the growing prevalence of chronic conditions range from individual-level disease management interventions (eg, clinical pharmacists) to community interventions that target population subgroups (eg, smoking bans in workplaces) or that target whole populations (eg, initiatives to promote fruit and vegetable consumption). Decision makers in communities, local public health agencies, and other settings can benefit from tools to support planning for chronic disease prevention programs and evaluation of the potential long-term impact of implemented interventions. Although short-term evaluations of public health interventions are useful for assessing what interventions were implemented, how many people were reached, and short-term changes in health behaviors or outcomes, a longer analysis time frame is needed to evaluate or project long-term changes in chronic disease outcomes.

The purpose of this study was to validate the current version of the Prevention Impacts Simulation Model (PRISM). PRISM was originally developed in 2005 to analyze the potential impacts of strategies to address cardiovascular disease (CVD) risk factors. PRISM is a population-level, mathematical model that synthesizes effect estimates from the literature and prevalence estimates from surveillance data sources such as the National Health and Nutrition Examination Survey (NHANES) to simulate health, mortality, and economic outcomes for the United States as a whole and for 6 community profiles defined by demographics related to population size, age group, and race and ethnicity. In this validation we focused on the nationally representative model.

Over the years, many disease prevention strategies and health and economic outcomes have been added to support the application of PRISM for policy planning and evaluation. For example, in 2016 a strategy to ban smoking in multi-unit housing was added to ensure that this priority strategy was among the options available to groups using PRISM for strategic planning and evaluation. Before 2020, PRISM was only available as part of Centers for Disease Control and Prevention (CDC) efforts to support chronic disease prevention program planning and evaluation for selected programs. However, the current version of PRISM, v3s3, is publicly available ( -simulation.cdc.gov/app/cdc/prism/#/) via CDC funding. As a result, additional community health planning groups can now use PRISM to inform decisions about chronic disease prevention and management strategies for their communities. Evidence of the validity of PRISM can increase the confidence of potential users that results of the model simulation can be usefully applied to inform policy decisions.

We used the recommended framework for model validation created by the International Society for Pharmoeconomics and Outcomes Research and the Society for Medical Decision Making for our analysis (1). This framework includes 5 types of validation for assessing a model: face validation, internal validation, cross validation, external validation, and predictive validation (1).

In 2019, we conducted validity checks of PRISM version 3q1a, focusing on face, internal, cross, and external validation. Predictive validation was not included because of the lack of real-world long-term (ie, 10 years or more) follow-up data on the impact of the health policies and interventions simulated in PRISM, which limits confidence in model results for prediction. However, we have sought to fill this gap by using out-of-sample surveillance data in external validation.

To conduct face validation of PRISM version 3q1a, CDC subject matter experts (SMEs) on heart disease, diabetes, smoking, and nutrition, physical activity, and obesity reviewed PRISM model structure, equations, input values, and data sources for inputs that varied most widely in the 1-way sensitivity analysis conducted as part of internal validation.

For internal validation, we had a secondary programmer review all model code and resolve all questions with the primary programmer. We also conducted 1-way sensitivity analyses to examine the impact on deaths, cardiovascular events, and medical costs of using the highest and lowest plausible values for each PRISM parameter (determined from the literature and SMEs) compared with the default parameter value. We used 1-way sensitivity analyses to examine how assumptions about specific parameters underlying modeled relationships drive results for all strategies modeled in PRISM. (A full list of the parameters examined in the sensitivity analysis is available upon request.) For this exercise, we moved all PRISM strategy levers to their maximum, which resulted in more than 15 million premature deaths averted from 2018 to 2040. We also examined sensitivity of individual strategy levers and results were similar (not reported). In sensitivity analysis, if the estimated impact of strategies does not change substantially based on the range of any specific parameters, then it demonstrates that variation in that parameter is not a concern for model results. One-way analyses assume that all inputs except the one under consideration remain at their default values.

In 2016, we conducted cross validation of the then-current version of PRISM (version 3q) by comparing simulated cardiovascular events and deaths with comparable results from 2 other simulation models for CVD: the CVD Policy Model (eg, Bibbins-Domingo et al [7]) and ModelHealth: CVD Microsimulation Model (eg, Dehmer et al [8]). This effort was part of a CDC study to explore the potential 5- and 10-year impacts of achieving Million Hearts (9) goals for aspirin use, blood pressure control, cholesterol control, sodium reduction, and smoking cessation and prevention (10). Results for cardiovascular events were similar across models. PRISM produced estimates for costs that were more conservative than the other 2 models.

We compared PRISM results with recent data from national surveys and surveillance systems. Previous validations analyzed PRISM version 3q output compared with national estimates for 1990 through 2010. Because more recent data became available for comparison with PRISM output, we extended the period for external validation through 2016 and assessed how well PRISM version 3q1a output matched surveillance data.

Table 1 shows the PRISM output measures included in external validation, comparable national data sources, and time periods included. (Detailed methods for how each measure was calculated are available upon request.) For each measure, we graphed the simulated PRISM outcomes and the corresponding surveillance data or weighted estimates from national surveys. Because PRISM analyzes adults with a prior CVD event (ie, post-CVD) separately from those with no prior CVD event (ie, non-CVD), we analyzed outputs separately for these subpopulations whenever possible. If trends in output measures in PRISM deviated substantially from surveillance data, we recalibrated the model to better track with surveillance data. We recalibrated only in the case of substantial deviations to avoid over-calibrating the model.

SMEs generally agreed with the model structure, parameter values, and data sources. SME review resulted in improvements in some documentation, and additional information was added to documentation about how parameter values were determined when there were gaps in available literature. Some parameters were updated based on SME input. Table 2 presents a summary of comments from SMEs and updates made to the model based on those comments. For example, because trans fats have largely been eliminated from the US food supply, these model parameters were updated. Additionally, SMEs shared more recent peer-reviewed published estimates of the effect of controlling prediabetes on diabetes onset. We therefore included these newer effect estimates in PRISM. We made these changes before conducting additional types of validation. In the second round of review as part of the CDC clearance process, we made the following model updates based on CDC SME comments: created separate levers for quality acute care and quality rehabilitation care, which had previously been combined; removed the use of aspirin for primary prevention of CVD based on updated evidence; and updated sources for physical activity in schools and child care levers.

3a8082e126