Segmentation of 4D images via space-time neural networks

Changjian Sun, Jayaram K. Udupa, Yubing Tong, Sanghun Sin, Mark Wagshul, Drew A. Torigian, Raanan Arens

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


Medical imaging techniques currently produce 4D images that portray the dynamic behaviors and phenomena associated with internal structures. The segmentation of 4D images poses challenges different from those arising in segmenting 3D static images due to different patterns of variation of object shape and appearance in the space and time dimensions. In this paper, different network models are designed to learn the pattern of slice-to-slice change in the space and time dimensions independently. The two models then allow a gamut of strategies to actually segment the 4D image, such as segmentation following just the space or time dimension only, or following first the space dimension for one time instance and then following all time instances, or vice versa, etc. This paper investigates these strategies in the context of the obstructive sleep apnea (OSA) application and presents a unified deep learning framework to segment 4D images. Because of the sparse tubular nature of the upper airway and the surrounding low-contrast structures, inadequate contrast resolution obtainable in the magnetic resonance (MR) images leaves many challenges for effective segmentation of the dynamic airway in 4D MR images. Given that these upper airway structures are sparse, a Dice coefficient (DC) of ~0.88 for their segmentation based on our preferred strategy is similar to a DC of >0.95 for large non-sparse objects like liver, lungs, etc., constituting excellent accuracy.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2020
Subtitle of host publicationBiomedical Applications in Molecular, Structural, and Functional Imaging
EditorsAndrzej Krol, Barjor S. Gimi
ISBN (Electronic)9781510634015
StatePublished - 2020
EventMedical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging - Houston, United States
Duration: Feb 18 2020Feb 20 2020

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging
Country/TerritoryUnited States

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Segmentation of 4D images via space-time neural networks'. Together they form a unique fingerprint.

Cite this