A Customizable Tool for Modeling the Health Impact of Different Colorectal Cancer Screening Strategies

We designed this model to support people in understanding the expected impact of different intervention strategies for improving colorectal cancer screening on long-term health outcomes in their local population or context. We believe this tool will be useful for a range of people making decisions about how to support colorectal cancer screening in their population, including but not limited to health administrators, providers, policymakers, state health departments, and payers.

A metamodel is a simplified version of a more complex simulation model. We used metamodeling to pre-run many different potential screening scenarios assuming different ranges of important model parameters. This allows users to be able to more quickly review the outputs specific to the particular scenarios and parameter values they enter.

Our previously developed simulation model is a validated, agent-based model designed to simulate the natural history and screening of colorectal cancer. It comprises three primary components: (1) adenoma (polyp) incidence and growth, (2) progression through precancerous and cancerous disease states, and (3) screening and surveillance dynamics. The model accepts population-level inputs, including demographic distributions and screening participation rates before, during, and after a five-year intervention period. It simulates individual life histories and produces detailed health outcomes, such as cancer diagnosis, cancer death, and life-years lost. Simulation outputs are used to generate metamodels that approximate the model's behavior and can be interfaced with via this metamodel decision tool. To train these metamodels, we simulated diverse combinations of screening levels with 5,000 stochastic replications per individual type in the base population. Each individual is characterized by age, race, and gender, enabling estimation of both individual-level and population-level outcomes.

Who is your population of interest? The tool allows you to enter your population size and the demographic characteristics of your population including age, race, and gender.

• What do I do if I don’t know all the requested population demographics for my local population? If you do not have data on your specific population, we have built in the demographic composition of the U.S. general population based on U.S. Census data.
• How do I run this model for different subpopulations? You can run the model for different subpopulations assuming different screening rates for these subgroups. You will need to separate your population into these subgroups and specify the screening rates specific to each subgroup. We allow users to stratify their population into subgroups to be able to generate estimates of health outcomes. This enables evaluation of screening interventions across heterogeneous populations. For example, one subgroup may consist of individuals with low screening levels prior to an intervention who experience substantial increases in screening, while another subgroup may include individuals with already high screening rates for whom the intervention yields more modest gains.
• To use this feature:
  
  • Toggle the "Multi-Population Analysis" option at the top of the tool.
  • Specify the percentage of your total population that each subgroup represents.
  • For each subpopulation, describe their demographics and their specific screening parameters, including baseline screening rates and any intervention scenarios you want to compare.
  • The model will calculate weighted results across all subpopulations based on their population shares.

What are the interventions you are considering? You can run and compare outcomes for up to 5 different scenarios (i.e., intervention strategies) at a time. Each scenario will represent a single strategy that you are considering to improve colorectal cancer screening in your population such as mailed FIT outreach or patient navigation. For each intervention, you will need to provide estimates of expected screening rates before, during, and after implementation of the intervention.

What is a baseline scenario? The baseline scenario is the comparison group for the other scenarios that you are exploring. This may be usual care or standard practice without any additional intervention in your population.

What are the before, during, and after parameters? The before, during, and after parameters represent the colorectal cancer screening rate in your population pre, during, and post implementation of each intervention strategy (i.e., scenario) that you are considering implementing. You will be asked to identify these screening rates for both FIT and colonoscopy. The before parameter represents your current screening rate by modality. For the during parameter, we assumed that intervention strategies would be implemented over a 5-year period. The during screening parameter, therefore, represents your expected screening rate after 5 years of intervention implementation. The after parameter represents your anticipated future screening rate after the 5-year intervention period has ended. ‍

• ‍How should I know what these screening parameters should be? For the before parameters, you can report your clinic or population’s current colorectal cancer screening rate by modality. To estimate the expected impact of different intervention strategies during the 5-year intervention period and in the future after the intervention period has ended, you can consult available evidence about the change in colorectal cancer screening associated with different intervention strategies. Alternatively, you could report what you think might be the best possible change in screening associated with each strategy.
‍

• You can choose from 6 different machine learning approaches (Linear Regression, Decision Tree, Random Forest, Support Vector Regression (SVR), Lasso, Ridge) to run your selected scenarios. Metamodels are estimation models.
• Linear models (Linear Regression, Lasso, Ridge) assume linear relationships between inputs and outputs, and the outputs are generated instantaneously.
• Tree-based models (Random Forest, Decision Tree) and SVR capture complex, non-linear patterns but may take longer to train.
• After training, the app displays R² scores for all models. R² scores are a measure of model accuracy and vary from 0 to 1. A score of 0 means that the model explains no variation seen in the real world, whereas a score of 1 means the model perfectly predicts all variation.
• Recommendation: Select the metamodel with the highest R² score, as it provides the most accurate prediction for your specific scenario.

The tool will provide the expected health impact associated with each scenario run based on the screening parameters and population characteristics specified. You will be able to view the number of colorectal cancer cases over the lifetime of the individuals in the population by scenario. The model will produce total cancer cases. It is important to note that increased screening will typically result in more cancer cases being diagnosed initially, but hopefully cancer cases will be averted over the long term. The model will also produce the number of deaths attributable to colorectal cancer. In addition, it will estimate the number of life-years. Increased screening will allow for cancers to be detected at earlier stages when the cancer is more treatable and will prevent other cancers from developing. The impact of shifting the stage of diagnosis can best be observed through the life-years outcome.

Click the "Download Results" button at the bottom of the Results tab. This will generate a comprehensive Excel file containing,

1. Your scenario comparison results (outcomes for baseline and intervention scenarios)
2. All input parameters used in the analysis (population size, screening rates, demographic distributions, etc.)
3. Model performance metrics (R² scores for the selected metamodel)
4. Timestamp and model type information

This tool currently models colorectal cancer screening using fecal immunochemical testing (FIT) and colonoscopy, but does not allow for additional screening modalities.

• You can contact systemstrengthening@unc.edu if you need assistance.