Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study

Abstract

The International Extranodal Lymphoma Study Group (IELSG)-37 is a prospective randomized trial assessing the role of consolidation mediastinal radiotherapy after immunochemotherapy to patients with newly diagnosed primary mediastinal large B-cell lymphoma (PMBCL). It is a positron emission tomography (PET) response-guided study where patients obtaining a complete metabolic response on an end-of-therapy PET-computed tomography (CT) scan evaluated by a central review are randomized to receive radiotherapy or no further treatment. The aims of this study were to measure agreement between reviewers reporting PET-CT scans for this trial and to determine the effect of training upon concordance rates. The review panel comprised 6 experienced nuclear physicians who read PET-CT scans using the 5-point Deauville scale. Interobserver agreement (IOA) was measured at 4 time points: after a blinded review of a "training set" of 20 patients with PMBCL from the previous IELSG-26 study (phase 1); after the first 10 clinical cases enrolled in the IELSG-37 (phase 2); and after 2 further groups of 50 (phase 3) and 40 clinical cases (phase 4). After feedback from the training set and the first 10 cases, a meeting was held to discuss interpretation, and a detailed set of instructions for the review procedure was agreed and acted upon. Between 2012 and 2014, the first 100 patients were reviewed. Using Deauville score 3 as the cutoff for a complete metabolic response, the overall IOA among the reviewers was good (Krippendorff α = 0.72.) The binary concordance between pairs of reviewers (Cohen κ) ranged from 0.60 to 0.78. The IOA, initially moderate, improved progressively from phase 1 to 4 (Krippendorff α from 0.53 to 0.81; Cohen κ from 0.35-0.72 to 0.77-0.87). Our experience indicates that the agreement among "expert" nuclear physicians reporting PMBCL, even using standardized criteria, was only moderate when the study began. However, agreement improved using a harmonization process, which included a training exercise with discussion of points leading to disagreement and compiling practical rules to sit alongside commonly adopted interpretation criteria.

More information

Identifiers

Catalogue record

Export record

ASCII CitationAtomBibTeXData Cite XMLDublin CoreDublin CoreEP3 XMLEndNoteHTML CitationHTML CitationHTML ListJSONMETSMODSMPEG-21 DIDLOpenURL ContextObjectOpenURL ContextObject in SpanRDF+N-TriplesRDF+N3RDF+XMLRIOXX2 XMLReferReference ManagerSimple Metadata

Altmetrics