LAUSR

Any modeling study such as the one by Prinja et al is supposed to mimic a real-world decision problem in a simplified manner. In order for a model to be trustworthy and reliable, it must produce results that pass the basic sniff test (“face validity”) of what happens in the real world. Second, the model description should contain all the details regarding the underlying assumptions so that other researchers can replicate the model findings. However, not only did Prinja et al fail to provide key model information, but their model also produced results that make little sense or are simply “impossible.” In other words, the model appeared to have failed the widely accepted norms of model transparency and validation, thereby making the study results very questionable. Following are our specific critiques. Although the authors claim to have used a combination of decision tree and Markov model, no model schematic was provided, and therefore the reader is left clueless as to how the authors conceptualized the decision problem, and how the decision problem was translated in the form of the model. If we are to go by the transition matrix of disease progression in Supporting Table 2, the model appeared to indicate that once patients have cancer, they either stay in the same cancer stage or move to the next worst stage but will never recover, which is simply unrealistic. The study did not report the size (number) of the hypothetical cohort that was simulated, nor was the number of Monte Carlo simulations presented, except that the latter number was reported to be >999. Was it 1000 or 100,000 or something else entirely? Prinja et al used utility values from an Indonesian study, completely ignoring the fact that population characteristics including the perception of health by the underlying population between the Punjab state of India and Indonesia might be very different. Such a “Band-Aid” approach to fill in data from disparate sources without proper justification can lead to misleading and potentially invalid results. More important, the Indonesian study had serious limitations in that it found higher utility for stage IV (0.77) than stage II (0.76) and III (0.71) disease, which could be due to the very small sample size (4 patients) of patients with stage IV disease or due to the use of a convenience sample. Prinja et al simply replaced the utility for stage IV by 0.66 (Supporting Table 2), without providing an explanation anywhere as to where they found this value. As is customary for any modeling study, the authors conducted a probabilistic sensitivity analysis using lognormal distribution for cost parameters, beta distribution for survival parameters, and uniform distribution for the remainder of the parameters. However, at no point in the text or in the supporting material were we able to find information regarding the specific distributional parameters used. This information is crucial for research reproducibility. The article by Prinja et al is riddled with numerous inconsistencies and red flags, which makes the entire modeling exercise very suspect. First, in Table 1, row 3, indicates total costs for no human papillomavirus (HPV) vaccination scenario: both the upper and lower limits were Indian National Rupees 85 million, which does not make sense. Second, in Table 2, numerous base-case results were beyond the bounds of the lower/upper limits. For example, total quality-adjusted life-years (discounted) for the base case of 3,400,922 was higher than the upper limit of 3,259,823! The incremental total life-years of 18,477 in the base case was not even within the lower (16,628) and upper (11,710) limits. It also should be noted that the lower limit was higher than the upper limit! These “impossible” results were repeated multiple times in Table 2, casting serious doubts over this model. Third, Prinja et al responded to Suman and Puliyel regarding the unprecedented rate of cervical cancer-related mortality of 99% by arguing that the result was to be interpreted as lifetime mortality as opposed to short-term mortality, and that the result was robust to different distributional assumptions for mortality distribution (eg, exponential and Weibul-Gompertz), as demonstrated in the sensitivity analyses that the authors performed. However, none of the specific parameter information for these sensitivity analyses was available either in the text or in the supporting materials. Moreover, we believe that the simple question regarding the incredulous mortality rate raised by Suman and Puliyel was left unanswered. Given these obvious and fatal flaws in the model by Prinja et al, we are concerned about the modeling methods used by the authors to “demonstrate that implementing HPV vaccination for adolescent girls in the Punjab state of India is very cost-effective.”