Artificial intelligence makes best microscopes better than ever

To observe the swift neuronal signals in a fish brain, scientists have started to use a technique called light-field microscopy, which makes it possible to image such fast biological processes in 3D. But the images are often lacking in quality, and it takes hours or days for massive amounts of data to be converted into 3D volumes and movies.

Now, EMBL scientists have combined artificial intelligence (AI) algorithms with two cutting-edge microscopy techniques – an advance that shortens the time for image processing from days to mere seconds, while ensuring that the resulting images are crisp and accurate. The findings are published in Nature Methods.

“Ultimately, we were able to take ‘the best of both worlds’ in this approach,” says Nils Wagner, one of the paper’s two lead authors and now a PhD student at the Technical University of Munich. “AI enabled us to combine different microscopy techniques, so that we could image as fast as light-field microscopy allows and get close to the image resolution of light-sheet microscopy.”

Although light-sheet microscopy and light-field microscopy sound similar, these techniques have different advantages and challenges. Light-field microscopy captures large 3D images that allow researchers to track and measure remarkably fine movements, such as a fish larva’s beating heart, at very high speeds. But this technique produces massive amounts of data, which can take days to process, and the final images usually lack resolution.

Light-sheet microscopy homes in on a single 2D plane of a given sample at one time, so researchers can image samples at higher resolution. Compared with light-field microscopy, light-sheet microscopy produces images that are quicker to process, but the data are not as comprehensive, since they only capture information from a single 2D plane at a time.

To take advantage of the benefits of each technique, EMBL researchers developed an approach that uses light-field microscopy to image large 3D samples and light-sheet microscopy to train the AI algorithms, which then create an accurate 3D picture of the sample.

“If you build algorithms that produce an image, you need to check that these algorithms are constructing the right image,” explains Anna Kreshuk, the EMBL group leader whose team brought machine learning expertise to the project. In the new study, the researchers used light-sheet microscopy to make sure the AI algorithms were working, Anna says. “This makes our research stand out from what has been done in the past.”

Robert Prevedel, the EMBL group leader whose group contributed the novel hybrid microscopy platform, notes that the real bottleneck in building better microscopes often isn’t optics technology, but computation. That’s why, back in 2018, he and Anna decided to join forces. “Our method will be really key for people who want to study how brains compute. Our method can image an entire brain of a fish larva, in real time,” Robert says.

He and Anna say this approach could potentially be modified to work with different types of microscopes too, eventually allowing biologists to look at dozens of different specimens and see much more, much faster. For example, it could help to find genes that are involved in heart development, or could measure the activity of thousands of neurons at the same time.

Next, the researchers plan to explore whether the method can be applied to larger species, including mammals.