Pushing the Limits of Fluorescence Microscopy with adaptive imaging and machine learning
Chan Zuckerberg Biohub
Fluorescence microscopy lets biologist see and understand the intricate machinery at the heart of living systems and has led to numerous discoveries. Any technological progress towards improving image quality would extend the range of possible observations and would consequently open up the path to new findings. I will show how modern machine learning and smart robotic microscopes can push the boundaries of observability. One fundamental obstacle in microscopy takes the form of a trade-of between imaging speed, spatial resolution, light exposure, and imaging depth. We have shown that deep learning can circumvent these physical limitations: microscopy images can be restored even if 60-fold fewer photons are used during acquisition, isotropic resolution can be achieved even with a 10-fold under-sampling along the axial direction, and diffraction-limited structures can be resolved at 20-times higher frame-rates compared to state-of-the-art methods. Moreover, I will demonstrate how smart microscopy techniques can achieve the full optical resolution of light-sheet microscopes — instruments capable of capturing the entire developmental arch of an embryo from a single cell to a fully formed motile organism. Our instrument improves spatial resolution and signal strength two to five-fold, recovers cellular and sub-cellular structures in many regions otherwise not resolved, adapts to spatiotemporal dynamics of genetically encoded fluorescent markers and robustly optimises imaging performance during large-scale morphogenetic changes in living organisms.
Royer first studied engineering in his native France and then obtained a master's degree in Artificial Intelligence, specializing in Cognitive Robotics, followed by a Ph.D. in Bioinformatics from the Dresden University of Technology in Germany. He then joined Gene Myers’ lab, first at HHMI's Janelia Farms and then at the Max Planck Institute of Molecular Cell Biology and Genetics, where he developed novel technology at the intersection of computer science and microscopy, including the first adaptive multi-view light-sheet microscope, which he developed in collaboration with Philipp Keller. As a group leader at CZ Biohub, Royer and his team are building ‘discovery machines’ that not only acquire image data, but also perform online processing, instant 3D visualization, adaptive imaging, and automated photo-manipulation. These integrated instruments bring together state of the art optics, robotics, machine learning, and image analysis with the aim of advancing beyond the automation of repetitive tasks and into the realm of actual automated scientific reasoning.