Inter-rater reliability of preprocessing EEG data: Impact of subjective artifact removal on associative memory task ERP results

Steven D. Shirk, Donald G. McLaren, Jessica S. Bloomfield, Alex Powers, Alec Duffy, Meghan B. Mitchell, Ali Ezzati, Brandon A. Ally, Alireza Atri

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


The processing of EEG data routinely involves subjective removal of artifacts during a preprocessing stage. Preprocessing inter-rater reliability (IRR) and how differences in preprocessing may affect outcomes of primary event-related potential (ERP) analyses has not been previously assessed. Three raters independently preprocessed EEG data of 16 cognitively healthy adult participants (ages 18-39 years) who performed a memory task. Using intraclass correlations (ICCs), IRR was assessed for Early-frontal, Late-frontal, and Parietal Old/new memory effects contrasts across eight regions of interest (ROIs). IRR was good to excellent for all ROIs; 22 of 26 ICCs were above 0.80. Raters were highly consistent in preprocessing across ROIs, although the frontal pole ROI (ICC range 0.60-0.90) showed less consistency. Old/new parietal effects had highest ICCs with the lowest variability. Rater preprocessing differences did not alter primary ERP results. IRR for EEG preprocessing was good to excellent, and subjective rater-removal of EEG artifacts did not alter primary memory-task ERP results. Findings provide preliminary support for robustness of cognitive/memory task-related ERP results against significant inter-rater preprocessing variability and suggest reliability of EEG to assess cognitive-neurophysiological processes multiple preprocessors are involved.

Original languageEnglish (US)
Article number322
JournalFrontiers in Neuroscience
Issue numberJUN
StatePublished - 2017


  • Artifacts
  • Inter-rater reliability
  • Memory
  • Preprocessing

ASJC Scopus subject areas

  • Neuroscience(all)


Dive into the research topics of 'Inter-rater reliability of preprocessing EEG data: Impact of subjective artifact removal on associative memory task ERP results'. Together they form a unique fingerprint.

Cite this