The University of Southampton
University of Southampton Institutional Repository

Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study

Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study
Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study

The International Extranodal Lymphoma Study Group (IELSG)-37 is a prospective randomized trial assessing the role of consolidation mediastinal radiotherapy after immunochemotherapy to patients with newly diagnosed primary mediastinal large B-cell lymphoma (PMBCL). It is a positron emission tomography (PET) response-guided study where patients obtaining a complete metabolic response on an end-of-therapy PET-computed tomography (CT) scan evaluated by a central review are randomized to receive radiotherapy or no further treatment. The aims of this study were to measure agreement between reviewers reporting PET-CT scans for this trial and to determine the effect of training upon concordance rates. The review panel comprised 6 experienced nuclear physicians who read PET-CT scans using the 5-point Deauville scale. Interobserver agreement (IOA) was measured at 4 time points: after a blinded review of a "training set" of 20 patients with PMBCL from the previous IELSG-26 study (phase 1); after the first 10 clinical cases enrolled in the IELSG-37 (phase 2); and after 2 further groups of 50 (phase 3) and 40 clinical cases (phase 4). After feedback from the training set and the first 10 cases, a meeting was held to discuss interpretation, and a detailed set of instructions for the review procedure was agreed and acted upon. Between 2012 and 2014, the first 100 patients were reviewed. Using Deauville score 3 as the cutoff for a complete metabolic response, the overall IOA among the reviewers was good (Krippendorff α = 0.72.) The binary concordance between pairs of reviewers (Cohen κ) ranged from 0.60 to 0.78. The IOA, initially moderate, improved progressively from phase 1 to 4 (Krippendorff α from 0.53 to 0.81; Cohen κ from 0.35-0.72 to 0.77-0.87). Our experience indicates that the agreement among "expert" nuclear physicians reporting PMBCL, even using standardized criteria, was only moderate when the study began. However, agreement improved using a harmonization process, which included a training exercise with discussion of points leading to disagreement and compiling practical rules to sit alongside commonly adopted interpretation criteria.

B-Lymphocytes, Humans, Mediastinal Neoplasms, Observer Variation, Positron Emission Tomography Computed Tomography, Prospective Studies, Journal Article, Review
0278-0232
548-553
Ceriani, Luca
5767bf0b-1df0-4479-90df-a3e0beb1c8b8
Barrington, Sally
94bd6716-f325-4e8e-9db3-8c35bbfbca69
Biggi, Alberto
392c3876-66a8-4dd1-802e-3cea650cc42e
Malkowski, Bogdan
70b51304-20e7-4247-b1fa-d848876bbe5e
Metser, Ur
3ceead4b-2f91-42d6-87cb-68cbc48a293e
Versari, Annibale
64a16bae-ea36-4859-83e6-9af2169847a6
Martelli, Maurizio
32f72970-6db0-45ec-9246-6d074b504192
Davies, Andrew
0fe6a40a-10d1-4ade-a7e6-d1dceb2470af
Johnson, Peter W
3f6068ce-171e-4c2c-aca9-dc9b6a37413f
Zucca, Emanuele
40bd5950-d74e-4169-a2ea-0bfc1058058e
Chauvie, Stéphane
f064b0cf-bde2-492e-bb0f-ed5c535a02fe
Ceriani, Luca
5767bf0b-1df0-4479-90df-a3e0beb1c8b8
Barrington, Sally
94bd6716-f325-4e8e-9db3-8c35bbfbca69
Biggi, Alberto
392c3876-66a8-4dd1-802e-3cea650cc42e
Malkowski, Bogdan
70b51304-20e7-4247-b1fa-d848876bbe5e
Metser, Ur
3ceead4b-2f91-42d6-87cb-68cbc48a293e
Versari, Annibale
64a16bae-ea36-4859-83e6-9af2169847a6
Martelli, Maurizio
32f72970-6db0-45ec-9246-6d074b504192
Davies, Andrew
0fe6a40a-10d1-4ade-a7e6-d1dceb2470af
Johnson, Peter W
3f6068ce-171e-4c2c-aca9-dc9b6a37413f
Zucca, Emanuele
40bd5950-d74e-4169-a2ea-0bfc1058058e
Chauvie, Stéphane
f064b0cf-bde2-492e-bb0f-ed5c535a02fe

Ceriani, Luca, Barrington, Sally, Biggi, Alberto, Malkowski, Bogdan, Metser, Ur, Versari, Annibale, Martelli, Maurizio, Davies, Andrew, Johnson, Peter W, Zucca, Emanuele and Chauvie, Stéphane (2017) Training improves the interobserver agreement of the expert positron emission tomography review panel in primary mediastinal B-cell lymphoma: interim analysis in the ongoing International Extranodal Lymphoma Study Group-37 study. Hematological Oncology, 35 (4), 548-553. (doi:10.1002/hon.2339).

Record type: Review

Abstract

The International Extranodal Lymphoma Study Group (IELSG)-37 is a prospective randomized trial assessing the role of consolidation mediastinal radiotherapy after immunochemotherapy to patients with newly diagnosed primary mediastinal large B-cell lymphoma (PMBCL). It is a positron emission tomography (PET) response-guided study where patients obtaining a complete metabolic response on an end-of-therapy PET-computed tomography (CT) scan evaluated by a central review are randomized to receive radiotherapy or no further treatment. The aims of this study were to measure agreement between reviewers reporting PET-CT scans for this trial and to determine the effect of training upon concordance rates. The review panel comprised 6 experienced nuclear physicians who read PET-CT scans using the 5-point Deauville scale. Interobserver agreement (IOA) was measured at 4 time points: after a blinded review of a "training set" of 20 patients with PMBCL from the previous IELSG-26 study (phase 1); after the first 10 clinical cases enrolled in the IELSG-37 (phase 2); and after 2 further groups of 50 (phase 3) and 40 clinical cases (phase 4). After feedback from the training set and the first 10 cases, a meeting was held to discuss interpretation, and a detailed set of instructions for the review procedure was agreed and acted upon. Between 2012 and 2014, the first 100 patients were reviewed. Using Deauville score 3 as the cutoff for a complete metabolic response, the overall IOA among the reviewers was good (Krippendorff α = 0.72.) The binary concordance between pairs of reviewers (Cohen κ) ranged from 0.60 to 0.78. The IOA, initially moderate, improved progressively from phase 1 to 4 (Krippendorff α from 0.53 to 0.81; Cohen κ from 0.35-0.72 to 0.77-0.87). Our experience indicates that the agreement among "expert" nuclear physicians reporting PMBCL, even using standardized criteria, was only moderate when the study began. However, agreement improved using a harmonization process, which included a training exercise with discussion of points leading to disagreement and compiling practical rules to sit alongside commonly adopted interpretation criteria.

Text
Training improves the interobserver agreement of the expert
Restricted to Repository staff only
Request a copy

More information

Accepted/In Press date: 13 July 2016
e-pub ahead of print date: 22 August 2016
Published date: December 2017
Additional Information: Copyright © 2016 John Wiley & Sons, Ltd.
Keywords: B-Lymphocytes, Humans, Mediastinal Neoplasms, Observer Variation, Positron Emission Tomography Computed Tomography, Prospective Studies, Journal Article, Review

Identifiers

Local EPrints ID: 417030
URI: https://eprints.soton.ac.uk/id/eprint/417030
ISSN: 0278-0232
PURE UUID: 65770461-0870-4e09-ba3b-f7143e3bc7ae
ORCID for Peter W Johnson: ORCID iD orcid.org/0000-0003-2306-4974

Catalogue record

Date deposited: 17 Jan 2018 17:30
Last modified: 15 Aug 2019 00:51

Export record

Altmetrics

Contributors

Author: Luca Ceriani
Author: Sally Barrington
Author: Alberto Biggi
Author: Bogdan Malkowski
Author: Ur Metser
Author: Annibale Versari
Author: Maurizio Martelli
Author: Andrew Davies
Author: Peter W Johnson ORCID iD
Author: Emanuele Zucca
Author: Stéphane Chauvie

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×