LAUSR

Importance In coordinated specialty care (CSC) settings for people with a first episode of psychosis, the development of reliable, validated individual-level prediction tools for key outcomes may be informative for shared clinician and client decision-making. Objective To develop an individual-level prediction tool using machine-learning methods that predicts a trajectory of education/work status or psychiatric hospitalization outcomes over a client's next year of quarterly follow-up assessments. Additionally, to visualize these predictions in a way that is informative to clinicians and clients. Design, Setting, and Participants Individual-level data were collected for all patients enrolled in the OnTrackNY program at enrollment and at quarterly follow-ups using standardized forms. The OnTrackNY program, a network of CSC sites in New York State, provides person-centered, recovery-oriented, and evidence-based psychosocial and pharmaceutical interventions to individuals aged 16 to 30 years with recent-onset (<2 years) nonaffective psychosis. Although data collection is ongoing, data for this study were collected from October 2013 to December 2018, and the time frame for analysis was July 2020 to May 2021. Data were separated into a training/cross-validation set to perform internally validated model development and a separate holdout test set (~20% of the sample) for external validation. Random probability forest models were developed to predict individual-level trajectories of outcomes. Exposures Forty-three individual-level demographic and clinical features collected at enrollment in OnTrackNY, 25 of which were time-varying and updated at quarterly follow-up assessments, and 13 site-level demographic and economic census variables. Main Outcomes and Measures Individual-level education and/or employment status and psychiatric hospitalization trajectories at quarterly follow-up periods across the first 2 years of CSC. Results The total study sample consists of 1298 individuals aged 16 to 30 years and included 341 women (26.3%), 949 men (73.1%), and 8 (<1%) with another gender. Prediction models performed well for 1-year trajectories of education/work across all validation sets, with areas under the receiver operating characteristic curve (AUCs) ranging from 0.68 (95% CI, 0.63-0.74) to 0.88 (95% CI, 0.81-0.96). Predictive accuracy for psychiatric hospitalization 3 months ahead reached AUC above 0.70; moreover, predictions of future psychiatric hospitalizations at 6 months and beyond were consistently poor, with AUCs below 0.60. Given the good externally validated performance for predicting education/work, a prototype interactive visualization tool displaying individual-level education/work trajectories and related features was developed. Conclusions and Relevance This study suggests that accurate prediction tools can be developed for outcomes in people with first-episode psychosis, which may help inform shared clinician/client decision-making. Future work should study the effectiveness of its deployment, including proper communication to inform shared clinician/client decision-making in the context of a learning health care system. At present, more work is needed to develop better performing prediction models for future psychiatric hospitalizations before any tool is recommended for this outcome.