LAUSR

We were interested to read the paper by Jensen et al. Published in Jul 2017 issue of NMR in Biomedicine. The aim of the authors was to investigate the test–retest reliability and feasibility of 6‐min J‐resolved magnetic resonance spectroscopy (MRS (scans for glutamate with healthy adolescents (n = 22) aged 12–14 years remaining in situ between successive scans. In secondary analyses, they evaluated the test–retest reliability for 12 other neurochemicals. MRS data were collected on a 3‐T SiemensTRIO Tim, whole‐body, clinical MR system using a 32‐channel, phased‐array design, radiofrequency (RF) head coil operating at 123 mhz for proton imaging and spectroscopy. The MRS data acquisitions were back‐to‐back, with no delay or re‐ acquisition of anatomical images between MRS scans. Glutamate and other metabolites were acquired from the rostral anterior cingulate cortex. For all 13 neurochemicals quantified, the percentage difference (% diff) for each individual subject was calculated between the two scans. Then, a group average of % diff reliability scores was calculated for each metabolite. To test reliability, Pearson correlations were computed between the two scans for each metabolite. Correlation results were interpreted exclusively for metabolites showing satisfactory variance between scan 1 and scan 2 (operationalized as <10% difference). Finally, a repeated‐measure analysis of variance (RMANOVA) tested main effects for ‘Time’ (Time 1, Time 2) for each metabolite that demonstrated <10% difference and showed sufficient test–retest reliability. As the authors mentioned, test–retest assessment of glutamate was reliable, as scores were less than 10% different (7.1 ± 4.2%), and glutamate values across scans were significantly correlated (Pearson r = 0.68, p < 10). Several other neurochemicals demonstrated satisfactory reliability, including choline (Cho) (7.4 ± 5.6%), glutathione (GSH) (8.6 ± 4.1%), myo‐inositol (mi) (6.5 ± 7.1%) and N‐acetylaspartate (NAA) (3.5 ± 3.6%), with test–retest correlations ranging from 0.74 to 0.95. A number of metabolites, however, did not demonstrate acceptable test–retest reliability using the current J‐resolved MRS sequence, ranging from 13.8 ± 13.7% (aspartate, Asp) to 45.9 ± 38.3% (glycine, Gly). Graduate students and bachelor's‐level research assistants administered the clinical interview after receiving 40 h of training, which included didactics, mock interviews and direct supervision. A clinically licensed psychologist (RPA) reviewed 20% of the interviews selected at random to assess inter‐rater reliability, and the Cohen's kappa coefficients were excellent (κ = 1.00). However, these results are not the most appropriate estimates to evaluate reliability. It is crucial to know that in term of reliability, using methods such as paired t‐test, repeated measures analysis of variance (RMANOVA), and Pearson's correlation for quantitative variables and Cohen's kappa coefficients for qualitative variables with more than two categories are among common mistakes in reliability analysis. It is important to note that, by comparing means (global average) using paired t.test or repeated‐measures analysis of variance (ANOVA), we can detect only the systematic change in concentrations between time 1 and 2, that is, a systematic increase or decrease. When the result of ANOVA is non‐significant, it does not mean that the concentrations are stable, because they could still be fluctuating in a non‐systematic way. Therefore, equality of the means has nothing to do with reliability. On the other hand, Pearson's correlation assumes that the relationships between variables are linear. So it measures a linear relationship, but fails to detect any non‐reliability (departure from the 45 line). Therefore, in case of using Pearson's correlation, high correlation coefficients are possible with far from reliability. Therefore, Pearson correlation coefficient, RMANOVA, and Cohen's kappa coefficients (except where the κ = 1.00) are not proper statistical methods for assessing repeatability (reliability). Briefly, for quantitative variables, the Intra Class Correlation Coefficient (ICCC) or Bland Altman Plot can be applied. For qualitative variables especially with more than two categories, weighted kappa is a good estimate. 5-7 As the authors pointed out in their conclusion, test–retest analyses suggest that clinically viable quantitative data can be obtained on standard MRI systems for glutamate, as well as the other metabolites, during short scan times in a traditionally challenging brain region. Such a conclusion can easily cause a misleading message in terms of reliability (precision). In conclusion, for reliability analysis, appropriate tests as well as correct interpretation should be applied. Otherwise, misdiagnosis and mismanagement cannot be avoided.