FPM-INR: Fourier ptychographic microscopy image stack reconstruction using implicit neural representations

1California Institute of Technology, 2University of Maryland, 3Rice University *Equal contribution
Optica, 2023

Framework Workflow

FPM-INR

Abstract

Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a high-resolution volumetric scene, impeding fast gigapixel-scale remote digital pathology. While deep learning approaches have been explored to address this challenge, existing methods poorly generalize to novel datasets and can produce unreliable hallucinations.

This work presents FPM-INR, a compact and efficient framework that integrates physics-based optical models with implicit neural representations (INR) to represent and reconstruct FPM image stacks. FPM-INR is agnostic to system design or sample types and does not require external training data. In our demonstrated experiments, FPM-INR substantially outperforms traditional FPM algorithms with up to a 25-fold increase in speed and an 80-fold reduction in memory usage for continuous image stack representations.

Image stack reconstructions

Tilted Human Blood Smear Stack

Thyroid Gland Lesion Stack

2D slice reconstruction

Siemens Star

We can also reconstruct 2D slices us FPM-INR method. In this demonstration, a Siemens Star resolution target with synthetic NA of 0.75 using 10X/0.25NA objective is presented. Use the image slide and click to drag to see the comparsion among the brightfield microscope, FPM, and FPM-INR. Our method shows equivalent performance as the FPM and superior resolution enhancement to brightfield image.


FPM-INR
Brightfield
FPM-INR
FPM



Sheep Blood Smear

Here we demonstrate an comple-field reconstruction using FPM and FPM-INR. The left image slider shows the amplitude comparison and the right shows the phase comparison. Use the image slide and click to drag to see the comparsion between FPM and FPM-INR.


FPM-INR
FPM
FPM-INR
FPM

BibTeX

@article{Zhou2023fpminr,
  author = {Haowen Zhou and Brandon Y. Feng and Haiyun Guo and Siyu (Steven) Lin and Mingshu Liang and Christopher A. Metzler and Changhuei Yang},
  journal = {Optica},
  keywords = {Biomedical imaging; Computer simulation; Deep learning; Neural networks; Phase retrieval; Systems design},
  number = {12},
  pages = {1679--1687},
  publisher = {Optica Publishing Group},
  title = {Fourier ptychographic microscopy image stack reconstruction using implicit neural representations},
  volume = {10},
  month = {Dec},
  year = {2023},
  url = {https://opg.optica.org/optica/abstract.cfm?URI=optica-10-12-1679},
  doi = {10.1364/OPTICA.505283}
}

Dataset DOI

CaltechDATA
DOI: https://doi.org/10.22002/7aer7-qhf77