Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation

Patrick Colley, Jess C. Mace, Madeleine R. Schaberg, Timothy L. Smith, Abtin Tabaee

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Objective Nasal endoscopy is integral to the evaluation of sinonasal disorders. However, prior studies have shown significant variability in the interrater agreement of nasal endoscopy interpretation among practicing rhinologists. The objective of the current study is to evaluate the interrater agreement of nasal endoscopy among otolaryngology residents from a single training program at baseline and following an educational intervention. Methods Eleven otolaryngology residents completed nasal endoscopy grading forms for eight digitally recorded nasal endoscopic examinations. An instructional lecture reviewing nasal endoscopy interpretation was subsequently provided. The residents then completed grading forms for eight different nasal endoscopic examinations. Interrater agreement among residents for the pre- and postlecture videos was calculated using the unweighted Fleiss' kappa (Kf) statistic and intraclass correlation agreement (ICC). Results Interrater agreement improved from a baseline level of fair (Kf range 0.268-0.383) to a posteducational level of moderate (Kf range 0.401-0.547) for nasal endoscopy findings of middle meatus mucosa, middle turbinate mucosa, middle meatus discharge, sphenoethmoid recess mucosa, sphenoethmoid recess discharge, and atypical lesions (ICC, P < 0.001). The baseline level of agreement for evaluation of nasal septum deviation was poor/fair and did not improve following educational intervention. Conclusions This study demonstrates a limited baseline level of interrater agreement of nasal endoscopy interpretation among otolaryngology residents. The interrater agreement for the majority of the characteristics that were evaluated improved after educational intervention. Further study is needed to improve nasal endoscopy interpretation.

Original languageEnglish (US)
Pages (from-to)2259-2265
Number of pages7
Issue number10
StatePublished - Oct 1 2015
Externally publishedYes


  • Nasal endoscopy
  • interrater agreement
  • otolaryngology
  • residency
  • sinusitis

ASJC Scopus subject areas

  • Otorhinolaryngology


Dive into the research topics of 'Impact of educational intervention on the interrater agreement of nasal endoscopy interpretation'. Together they form a unique fingerprint.

Cite this