Be a Part of the Open-Source Process

Models developed as part of IVI’s Open-Source Value Platform (OSVP) serve as a hands-on “laboratory” for improving methods, resolving scientific questions, and engaging in an ongoing public discussion about value in healthcare. This iterative process is made possible by an ongoing cycle of public feedback, expert review, revision, and re-release.

Everyone with a stake in how treatment value is measured and paid for in the U.S. healthcare system has a voice in the open-source development of IVI models. Feedback is welcome and encouraged, from the highly detailed suggestions, such as how to fix a bug in the model’s source code, to the big picture – like identifying important determinants of value that are currently missing from the model, for example, or proposing alternative approaches to value-based decision-making.

There are multiple ways to provide feedback and make recommendations to improve the value tools:

  • Submit Formal Comments During Open Comment Periods: Written comments, methodological feedback, and recommendations for additions or changes should be submitted in Word format to research@thevalueinitiative.org. All submitted comments will be made publicly available. Please include organizational letterhead, if applicable.
  • Modify the Source Code: To contribute directly to the development of the underlying model, click here access the source code and submit changes on the IVI GitHub page.

All feedback submitted during open comment periods will be reviewed by a panel of independent subject matter experts. The role of the Technical Expert Panel is to synthesize feedback and develop a prioritized list of recommended changes to the modeling platform, as well as areas for further research.

Based on the recommendations from the previous step, IVI researchers will implement changes and develop a “roadmap” for future revisions. As needed, IVI partners with patients and patient advocates, providers, external scientific teams, and others to implement changes.

A new version of each IVI value tool and the underlying model is released after recommended changes have been completed. These model releases are accompanied by documentation that includes the recommendations of the Technical Expert Panel, a summary of changes made to the model, and draft “roadmap” for future development. At this time, a new period of public feedback will begin, and the process will repeat.

The IVI Open-Source Development Process

A Fully Transparent Process

Transparency is a key principal of the Open-Source Value Platform and essential to the open-source development process. In addition to making all modeling code and methods available for public download and use, IVI makes the feedback, review, and revision process fully transparent. All feedback received on IVI’s OSVP models is made publicly available following the end of the public comment period. Similarly, the results of the Technical Expert Panel’s review and recommendations are publicly released, and IVI’s planned revisions in response are shared ahead of the subsequent model release.

Feedback on Current Models

  • IVI-RA Pre-Release Feedback: IVI received helpful feedback from a wide range of stakeholders prior to release, including detailed review of model methods and interface design. IVI received feedback in a variety of formats, including both informal discussions with patient advocates and clinicians, as well as formal written comments. View pre-release comments here.
  • Comments on IVI-RA Initial Release: IVI held a three-month open feedback period on the initial IVI-RA model release, ending February 18, 2018. These comments, which represent a range of perspectives, are the basis for upcoming model revisions and will inform future iterations of both the IVI-RA model and future OSVP models. View consolidated public comments here.
  • TEP Review of IVI-RA Initial Release and Planned Revisions: Public feedback received on the initial release of the IVI-RA model was reviewed by the RA Technical Expert Panel (TEP), an objective third party comprised of leaders in health economics, epidemiology, rheumatology, and patient communities. Based on public input, the TEP provided recommendations for revisions for version 2.0 of the IVI-RA model. View detailed TEP report here.

  • Comments on IVI-NSCLC Initial Release: IVI held an open feedback period for the initial release of the IVI-NSCLC model between January 31 and April 1, 2019. Similar to the IVI-RA model, the comments received will serve as a basis for future revisions of the IVI-NSCLC model. View the public comments here.