Letter

Re: The Benefits of High Relaxivity for Brain Tumor Imaging: Results of a Multicenter Intraindividual Crossover Comparison of Gadobenate Dimeglumine with Gadoterate Meglumine (The BENEFIT Study)

E. Lancelot, B. Piednoir, P. Desché
Guerbet
Roissy CdG Cedex, France

We have read with interest the publication by Vaneckova et al¹ reporting the results of a clinical study that assessed the diagnostic performances of 2 gadolinium-based contrast agents (GBCAs) used for brain tumor imaging. The authors performed a multicentric, prospective, randomized, intraindividual, crossover, 2-arm study. The objective of Arm 1 was to demonstrate the superiority of a full dose (0.1 mmol/kg) of gadobenate dimeglumine over the same dose of gadoterate meglumine, whereas in Arm 2, the aim was to ascertain whether a half dose (0.05 mmol/kg) of gadobenate provides diagnostic information similar to that of a full dose of gadoterate. GBCA administrations and image analyses were performed in a blinded manner. The primary end point was the overall diagnostic preference of the readers for one GBCA over the other. In Arm 1, a significant superiority was shown in favor of gadobenate, and in Arm 2, no significant differences could be found between the 2 GBCAs. The authors concluded that when administered at the approved dose of 0.1 mmol/kg, gadobenate is superior to gadoterate for qualitative and quantitative assessment of brain lesions, and that a half dose of the former agent is equivalent to a full dose of the latter. However, we consider that some biases limit the interpretation of the results and even lead to wrong assertions.

First, the statistical analysis was not adapted to the objectives of the study. To compute the sample size in each arm, the authors assumed that no difference in overall diagnosis preference would be found between the 2 GBCAs in half of the patients. In the other half, they hypothesized that the preference would be in favor of gadobenate in 80% of the patients who received the full dose (Arm 1) and in 75% of those who received the half dose (Arm 2). Then they applied the Wilcoxon signed rank test to demonstrate the superiority of gadobenate in both arms. The results showed a significant preference for this GBCA in Arm 1 but not in Arm 2. However, in Arm 1, the agreement among the 3 readers reached only 50.8%, with a κ value of 0.273. According to Landis and Koch,² this is a moderate level of agreement, and it casts some doubt on the robustness of the interpretations. The second arm was clearly not designed as an equivalence or a noninferiority trial, as defined in the “Consolidated Standards of Reporting Trials” statement,³ and failure to show a difference should not have been interpreted as an equivalence between both GBCAs. Therefore, when the authors concluded that “a half dose of gadobenate (0.05 mmol/kg body weight) is equivalent to a full dose (0.1 mmol/kg body weight) of gadoterate,”¹ they obviously made a biased interpretation of the results. The comparison in Arm 2 simply failed because the hypothesis of superiority was not met.

Second, the number of lesions subjected to signal intensity measurements with the T1-weighted gradient-echo (T1GRE) sequence differed from that of the T1-weighted spin-echo (T1SE) sequence. Most surprising, fewer lesions were considered with the T1GRE sequence, though they were all larger than 5 mm: In Arm 1, 63, 66, and 54 lesions were assessed by readers 1, 2, and 3 in T1SE and 60, 61, and 51 lesions in T1GRE; in Arm 2, 84, 89, and 78 lesions were assessed in T1SE and 82, 85, and 75 lesions in T1GRE. This discrepancy between sequences may have created a bias in the analysis of the images. As both GBCAs assessed the same number of lesions, it is likely that the choice of sequence is more important than differences in relaxivity between GBCAs.

In conclusion, the design of this multicenter randomized clinical trial had some important weaknesses that affected the comparability between the 2 GBCAs. Some of the conclusions are not supported by the results, especially the assumed equivalence of the half dose of gadobenate. As for the full dose, the low interreader agreement shows the variability of the interpretation of a qualitative end point such as the diagnostic preference of one GBCA over another. More important is the clinical impact of the diagnosis on patient management. Unfortunately, this end point was not assessed in the present study.

Conflict of Interest

The authors of this letter declare that they currently have a conflict of interest because they are employed by Guerbet.

References

1. Vaneckova M, Herman M, Smith MP, et al. The benefits of high relaxivity for brain tumor imaging: results of a multicenter intraindividual crossover comparison of gadobenate dimeglumine with gadoterate meglumine (the BENEFIT study). AJNR Am J Neuroradiol 2015;36:1589–98
2. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74
3. Piaggio G, Elbourne DR, Pocock SJ, et al. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 Statement. JAMA 2012;308:2594–604

Reply

A. Spinazzi, G. Pirovano, N. Shen
Global Medical and Regulatory Affairs
Bracco Diagnostics
Monroe, New Jersey

M.A. Kirchin
Global Medical and Regulatory Affairs
Bracco Imaging
Milan, Italy

We thank Drs Lancelot, Piednoir, and Desché for their interest in our work and their comments regarding our recent article.¹ However, we disagree strongly with the critique presented in their letter. The necessity to comply with word count limitations when preparing manuscripts for publication means that the description of the statistical methodology is often too brief. We now address the points raised.

Statistical Analysis and Sample Size

The study design and analysis described in our article¹ are similar to the methodology used in many previous intraindividual comparative studies.^{2⇓⇓⇓⇓⇓⇓–9} The primary study end point was the overall diagnostic preference of each of 3 readers for one gadolinium-based contrast agent (GBCA) over the other. Other qualitative end points (determinations of lesion border delineation, definition of disease extent, visualization of lesion internal morphology, and lesion contrast enhancement) are accepted clinically relevant parameters that can directly impact patient management decisions and surgical planning, particularly for patients with glial tumors in whom macroscopically complete surgical removal is associated with improved prognosis and longer patient survival, and those with metastases, for whom determination of the precise number, size, and location of lesions can aid selection of the most appropriate treatment option.^{3,6⇓⇓–9} Image assessment was performed by comparing images from the 2 MR imaging examinations side-by-side, with the readers blinded to the contrast agent used and all clinical information. Each reader expressed preference for examination 1 or 2 or determined that the 2 examinations were equal. The resulting data for each reader were 1 observation per patient (ie, 1 paired sample datum with an ordinal scale). Nonparametric analysis with a 2-sided Wilcoxon signed rank test is the appropriate statistical analysis method for assessment of overall diagnostic preference. The distribution of the preference between the 2 examinations was also tested by using a 1-sample χ² test for equal proportions. The results obtained were similar to those from the Wilcoxon signed rank test. This analysis was not included in the article due to the word restrictions.

The results for Study Arm 1 revealed a highly significant (P < .0001) preference for gadobenate over gadoterate at an equivalent dose for each reader (Fig 1). Variation of measurements between readers is expected in a fully blinded read setting. Figure 1 shows that all 3 readers were in high agreement and consistent, especially concerning the very few assessments in which gadoterate was preferred over gadobenate (1.6%–3.2% of patients across 3 readers). The major reason for the 50.8% agreement among readers was the differential percentage of preferences for gadobenate across the 3 readers (49.2%, 82.3%, 69.4%). As already discussed in our article,¹ a κ value of 0.273 was due to the skewed distribution of preferences (very few preferences for gadoterate). Feinstein et al¹⁰ demonstrated clearly that a low κ can result from a substantial imbalance in marginal totals.

The results for Study Arm 2 revealed no statistically significant differences between gadoterate at 0.1 mmol/kg and gadobenate at 0.05 mmol/kg of body weight.¹ In the abstract, we concluded, “No meaningful differences were recorded between 0.05 mmol/kg gadobenate and 0.1 mmol/kg gadoterate.” We understand from a statistical point of view that equivalence cannot be claimed if the test hypothesis is not prospectively defined as “noninferiority.” However, a conclusion of “no statistically significant difference” between treatments simply means that the evidence that the 2 treatments lead to different outcomes is not strong enough. As can readily be seen inFig 2, all 3 readers determined that the images from most patients were diagnostically equal (ie, no diagnostic preference between the images with half-dose gadobenate and full-dose gadoterate).