Digital evolution: how computing and AI are driving innovation in microscopy and advancing the life sciences
Biologists and other life scientists depend on microscopy to visualize cells and tissues in great detail in biological samples. To characterize samples to understand biological processes, including disease mechanisms, researchers are increasingly using a technique of multiplex or multicolor microscopy. This allows you to color different parts of the sample a different color – for example, the nucleus of a cell can be blue while the cell membrane is colored red.
Because each fluorophore has a characteristic emission spectrum – the range of wavelengths in which it emits light – the choice of fluorophores is critical when using different dyes at the same time to study interactions. As most fluorophores have a broad emission spectrum, it is important when using two or more fluorophores to select those where the overlap in emission spectra is minimal. If there is overlap, their signals will interfere with each other in a phenomenon known as “crosstalk”, making the resulting data difficult to interpret.
It is therefore vital that the fluorophores are well distinguished to obtain optimal results during multiplexing. Examples of where this is needed include studying different immune cells in relation to key biomarkers in immuno-oncology studies, and discrete visualization of multiple proteins to identify different types of neurons in complex synaptic networks in neuroscience studies (see Figure 1).1
Figure 1: Adult rat brain visualized by multicolor or multiplex microscopy. Neuronal cells were stained green with the fluorophore, Alexa Fluor488; astrocytes were stained red with glial fibrillary acidic protein (GFAP) staining and cell nuclei were stained blue with 4′,6-diamidino-2-phenylindole (DAPI). Image courtesy of Professor En Xu, Institute of Neuroscience and Department of Neurology, Second Affiliated Hospital, Guangzhou Medical University, China.1
Algorithmic solutions offering new possibilities for interpreting data
To help overcome the challenges of multiplex microscopy, automated tools have been developed to help researchers algorithmically “demix” spectral ranges – yielding faster, smarter and higher quality data and providing new possibilities for interpretation of image data. For example, Drs. Francesco Cutrale and Scott E.Fraser at the University of Southern California (USC) Center for Translational Imaging have synthesized a new way to combine two existing methods and intelligently automate it using simple algorithms to allow scientists to acquire, instantly store and analyze images produced using multiple fluorescent markers, in one go.2.3
The first method involved is hyperspectral imaging, which was originally developed for remote sensing by aircraft flying over the earth or satellites flying over the globe. This type of imaging incorporates the additional dimension of wavelength, in that it simultaneously captures different wavelengths of light in a large number of channels, rather than sequentially capturing monochrome images respectively in a more small number of channels.3
While this approach promised a better way to capture high-resolution multicolor images, there were challenges in repurposing technology for microscopy, including the light source, which in the original hyperspectral imaging method was the sun. The much weaker light signal provided with microscopy meant having to deal with low signal-to-noise ratios in fluorescence. Fluorescence hyperspectral imaging for microscopy has also been affected by speed and a limited photon budget.3
But these hurdles haven’t deterred Professor Fraser, who managed to successfully implement hyperspectral sensing for microscopes 20 years ago.3 He used complex mathematics to disentangle the signals and separate the contributions made by each type of spectral emission from the biological sample stained with multiple fluorophores. More recently, Dr. Cutrale has found a robust algorithm that is not only resilient to the types of noise encountered using hyperspectral microscopy, but is also simple enough to describe not just the single pixel – which may be more susceptible to the effects of noise – but also the complete spectral composition of the entire sample.3
Removing noise from hyperspectral microscopy
This is where the second method comes in. Like hyperspectral imaging, phasor analysis has also been around for decades and is already well established for fluorescence lifetime imaging (FLIM). But that hadn’t been applied to hyperspectral microscopy, that is, only three years ago when Dr. Cutrale started working on hyperspectral phasors.3 He found it to be a powerful and effective tool for removing noise from hyperspectral microscopy data.
But one hurdle remained: Researchers needed to learn about phasor analysis at an expert level in order to truly understand how to manipulate the phasor to demix signals and effectively denoise their data. To overcome this, Dr. Cutrale integrated a hybrid linear demixing algorithm and the partial automation of standard algorithms with the versatility and sensitivity of the phaser to demix the signals and ended up with a solution that was faster, more sensitive and much easier to use. .
In fact, the speed, sensitivity, and utility gains of this method combined, along with fluorescent hyperspectral microscopy and phasor analysis, allow researchers to capture and analyze their images in real time, meaning that any failures can be corrected immediately by capturing new images of the same sample, which would be much more difficult, if not impossible, if analysis had to be performed on the data much later – after the dyes had lost their fluorescence.2.3
Machine learning to map the subcellular distribution of human proteins
The rapid increase in computing power is also helping to innovate in the life sciences through advanced microscopy in the Human Protein Cell Atlas (HPA) program. dr. Emma Lundbergprofessor of cell biology proteomics at KTH Royal Institute of Technology in Sweden and director of HPA, develops machine learning (ML) models to help map the subcellular distribution of most human proteins and study their movements and interactions in real time.
The ML algorithms developed by his team help improve image segmentation in confocal microscopy and enable much more efficient data processing and analysis, including image segmentation into multiple sets of pixels,4.5 recognize patterns of protein distribution without bias and identify even subtle changes in cell morphology.4 Similarly, rare cell types can also be identified in these ML-based analyses.4
These ML models can also incorporate spatial information in a format that can be integrated with other types of molecular characterization, for example, with proteomic data or single cell sequencing data.4 Real-time analysis and frequent time-lapse imaging also allow the team to observe dynamic events in cells, including rare events in cells that they can selectively image with light-powered microscopy. ‘IA.4
Expand access to microscopy by eliminating the complexity of multispectral imaging
Although hyperspectral unmixing had been used for some time in satellite images that used the sun as a light source, it proved difficult to develop this method for microscopy, which was hampered by a very low signal-to-noise ratio in fluorescence. Ultimately, it took the combination of several methods, including phasor-based analysis and automated linear unmixing to arrive at a fast and reliable microscopy technique for hyperspectral unmixing in a “plug and play” format. accessible to a wider range of scientists.6 This is gentle on the sample as only one image exposure is needed.
With automated hybrid spectral unmixing, multicolor fluorescent imaging is now easier and faster, providing scientists with an ideal solution for digitizing large samples or capturing fast dynamic processes in living cells.6 Combined with ML algorithms, these methods allow researchers to extract knowledge from their microscopy samples while they are still sitting at the microscope with their sample, focusing on getting results instead of understanding their microscope. which frees them to collect better data and perform better science.
1. Pelzer P. Multicolor Microscopy: The Importance of Multiplexing. Leica Microsystems. https://www.leica-microsystems.com/science-lab/multicolor-microscopy-the-importance-of-multiplexing/. Published January 10, 2022. Accessed January 19, 2022.
2. Polakovic G. From detecting lung cancer to detecting counterfeit money, this new imaging technology could have countless uses. USC stem cell. https://stemcell.keck.usc.edu/from-detecting-lung-cancer-to-spotting-counterfeit-money-this-new-imaging-technology-could-have-countless-uses/. Published February 5, 2020. Accessed February 11, 2022.
3. Cutrale F, Trivedi V, Trinh L, et al. Hyperspectral phasor analysis enables multiplexed 5D live imagery. National Methods . 2017. https://doi.org/10.1038/nmeth.4134
4. Lundberg E, Leica Microsystems Corporate Communications. Application of AI and machine learning to microscopy and image analysis. Leica Microsystems. https://www.leica-microsystems.com/science-lab/applying-ai-and-machine-learning-in-microscopy-and-image-analysis/. Published January 10, 2022. Accessed January 21, 2022.
5. Petoukhov E. Use of machine learning in image analysis in microscopy. Leica Microsystems. https://www.leica-microsystems.com/science-lab/using-machine-learning-in-microscopy-image-analysis/. Published January 10, 2022. Accessed January 24, 2022.
6. Amon J, Laskey P. FluoSync – a fast and gentle method for untangling multicolor widefield fluorescence images. White paper. Leica Microsystems. https://go.leica-ms.com/FluoSync. Published January 10, 2022. Accessed February 11, 2022.