Saga Helgadottir – Soft Matter Lab

Roadmap on Deep Learning for Microscopy on ArXiv

Spatio-temporal spectrum diagram of microscopy techniques and their applications. *(Image by the Authors of the manuscript.)*

Roadmap on Deep Learning for Microscopy
Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C.D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu, Li-Yu Yu, Sixian You, Yongtao Liu, Maxim A. Ziatdinov, Sergei V. Kalinin, Arlo Sheridan, Uri Manor, Elias Nehme, Ofri Goldenberg, Yoav Shechtman, Henrik K. Moberg, Christoph Langhammer, Barbora Špačková, Saga Helgadottir, Benjamin Midtvedt, Aykut Argun, Tobias Thalheim, Frank Cichos, Stefano Bo, Lars Hubatsch, Jesus Pineda, Carlo Manzo, Harshith Bachimanchi, Erik Selander, Antoni Homs-Corbera, Martin Fränzl, Kevin de Haan, Yair Rivenson, Zofia Korczak, Caroline Beck Adiels, Mite Mijalkov, Dániel Veréb, Yu-Wei Chang, Joana B. Pereira, Damian Matuszewski, Gustaf Kylberg, Ida-Maria Sintorn, Juan C. Caicedo, Beth A Cimini, Muyinatu A. Lediju Bell, Bruno M. Saraiva, Guillaume Jacquemet, Ricardo Henriques, Wei Ouyang, Trang Le, Estibaliz Gómez-de-Mariscal, Daniel Sage, Arrate Muñoz-Barrutia, Ebba Josefson Lindqvist, Johanna Bergman
arXiv: 2303.03793

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Dynamic live/apoptotic cell assay using phase-contrast imaging and deep learning on bioRxiv

Phase-contrast image before virtual staining. *(Image by the Authors.)*

Dynamic live/apoptotic cell assay using phase-contrast imaging and deep learning
Zofia Korczak, Jesús Pineda, Saga Helgadottir, Benjamin Midtvedt, Mattias Goksör, Giovanni Volpe, Caroline B. Adiels
bioRxiv: https://doi.org/10.1101/2022.07.18.500422

Chemical live/dead assay has a long history of providing information about the viability of cells cultured in vitro. The standard methods rely on imaging chemically-stained cells using fluorescence microscopy and further analysis of the obtained images to retrieve the proportion of living cells in the sample. However, such a technique is not only time-consuming but also invasive. Due to the toxicity of chemical dyes, once a sample is stained, it is discarded, meaning that longitudinal studies are impossible using this approach. Further, information about when cells start programmed cell death (apoptosis) is more relevant for dynamic studies. Here, we present an alternative method where cell images from phase-contrast time-lapse microscopy are virtually-stained using deep learning. In this study, human endothelial cells are stained live or apoptotic and subsequently counted using the self-supervised single-shot deep-learning technique (LodeSTAR). Our approach is less labour-intensive than traditional chemical staining procedures and provides dynamic live/apoptotic cell ratios from a continuous cell population with minimal impact. Further, it can be used to extract data from dense cell samples, where manual counting is unfeasible.

Press release on Extracting quantitative biological information from bright-field cell images using deep learning

Virtually-stained generated image for lipid-droplet.

The article Extracting quantitative biological information from bright-field cell images using deep learning has been featured in a press release of the University of Gothenburg.

The study, recently published in Biophysics Reviews, shows how artificial intelligence can be used to develop faster, cheaper and more reliable information about cells, while also eliminating the disadvantages from using chemicals in the process.

Here the links to the press releases on Cision:
Swedish: Effektivare studier av celler med ny AI-metod
English: More effective cell studies using new AI method

Here the links to the press releases in the News of the University of Gothenburg:
Swedish: Effektivare studier av celler med ny AI-metod
English: More effective cell studies using new AI method

Saga Helgadottir joins as postdoc the Soft Matter Lab

Saga Helgadottir starts her postdoc at the Physics Department of the University of Gothenburg on 1st September 2021.

Saga has a PhD degree in Physics from the University of Gothenburg, Sweden.

During her postdoc, Saga will continue her work on developing tools using deep learning to solve problems in image analysis and medical diagnosis.

Enhanced prediction of atrial fibrillation and mortality among patients with congenital heart disease using nationwide register-based medical hospital data and neural networks published in European Heart Journal – Digital Health

Neural network prediction of mortality and atrial fibrillation. (Image taken from the article’s graphical abstract.)

Enhanced prediction of atrial fibrillation and mortality among patients with congenital heart disease using nationwide register based medical hospital data and neural networks
Kok Wai Giang, Saga Helgadottir, Mikael Dellborg, Giovanni Volpe, Zacharias Mandalenakis
European Heart Journal – Digital Health (2021)
doi: 10.1093/ehjdh/ztab065

Aims: To improve short-and long-term predictions of mortality and atrial fibrillation (AF) among patients with congenital heart disease (CHD) from a nationwide population using neural networks (NN).

Methods and results: The Swedish National Patient Register and the Cause of Death Register were used to identify all patients with CHD born from 1970 to 2017. A total of 71 941 CHD patients were identified and followed-up from birth until the event or end of study in 2017. Based on data from a nationwide population, a NN model was obtained to predict mortality and AF. Logistic regression (LR) based on the same data was used as a baseline comparison. Of 71 941 CHD patients, a total of 5768 died (8.02%) and 995 (1.38%) developed AF over time with a mean follow-up time of 16.47 years (standard deviation 12.73 years). The performance of NN models in predicting the mortality and AF was higher than the performance of LR regardless of the complexity of the disease, with an average area under the receiver operating characteristic of >0.80 and >0.70, respectively. The largest differences were observed in mortality and complexity of CHD over time.

Conclusion: We found that NN can be used to predict mortality and AF on a nationwide scale using data that are easily obtainable by clinicians. In addition, NN showed a high performance overall and, in most cases, with better performance for prediction as compared with more traditional regression methods.

Extracting quantitative biological information from brightfield cell images using deep learning featured in AIP SciLight

The article Extracting quantitative biological information from brightfield cell images using deep learning
has been featured in: “Staining Cells Virtually Offers Alterative Approach to Chemical Dyes”, AIP SciLight (July 23, 2021).

Scilight showcases the most interesting research across the physical sciences published in AIP Publishing Journals.

Scilight is published weekly (52 issues per year) by AIP Publishing.

Extracting quantitative biological information from bright-field cell images using deep learning published in Biophysics Reviews

Extracting quantitative biological information from bright-field cell images using deep learning
Saga Helgadottir, Benjamin Midtvedt, Jesús Pineda, Alan Sabirsh, Caroline B. Adiels, Stefano Romeo, Daniel Midtvedt, Giovanni Volpe
Biophysics Rev. 2, 031401 (2021)
arXiv: 2012.12986
doi: 10.1063/5.0044782

Quantitative analysis of cell structures is essential for biomedical and pharmaceutical research. The standard imaging approach relies on fluorescence microscopy, where cell structures of interest are labeled by chemical staining techniques. However, these techniques are often invasive and sometimes even toxic to the cells, in addition to being time-consuming, labor-intensive, and expensive. Here, we introduce an alternative deep-learning-powered approach based on the analysis of bright-field images by a conditional generative adversarial neural network (cGAN). We show that this approach can extract information from the bright-field images to generate virtually-stained images, which can be used in subsequent downstream quantitative analyses of cell structures. Specifically, we train a cGAN to virtually stain lipid droplets, cytoplasm, and nuclei using bright-field images of human stem-cell-derived fat cells (adipocytes), which are of particular interest for nanomedicine and vaccine development. Subsequently, we use these virtually-stained images to extract quantitative measures about these cell structures. Generating virtually-stained fluorescence images is less invasive, less expensive, and more reproducible than standard chemical staining; furthermore, it frees up the fluorescence microscopy channels for other analytical probes, thus increasing the amount of information that can be extracted from each cell.

Saga Helgadottir defended her PhD Thesis in Physics on June 16, 2021. Congrats!

Saga Helgadottir defended her PhD Thesis in Physics on June 16, 2021. Congrats!

The disputation took place at 9 a.m. digitally via Zoom. A link to the Zoom meeting was published the day before dissertation on the GU website.

Title: Deep Learning Applications – From image analysis to medical diagnosis

Abstract:
Deep learning is a subcategory of machine learning and artificial intelligence. Instead of using explicit rules to perform a desired task as in standard algorithmic approaches, machine-learning algorithms autonomously learn from data to determine the rules for the task at hand. The idea of deep learning has been around since the 1950s but was for a long time limited by available computational power and amount of training data. Once overcome these problems, in recent years, deep learning has made great advances in solving various problems.

In this thesis, I show how deep learning can be applied in image analysis and medical diagnosis, while outperforming standard algorithmic methods and simpler machine-learning methods. I begin with showing that a convolutional neural network trained with simulated particle images is able to track experimental single particles, even in poor illumination conditions. I then show how this inspired the development of an all-in-one software to design, train and validate deep-learning solutions for digital microscopy, from particle tracking and characterization in 2D and 3D to the segmentation, characterization and counting of biological cells and image transformation. I show that this software package can be further used to develop a generative adversarial neural network to virtually stain brightfield images of cells, replacing the traditional chemical staining for a downstream analysis of biological features. I then move on from applications in microscopy and image analysis to show the potential of deep learning in medical diagnosis. I show that dense neural networks perform better than simpler machine-learning algorithm and the clinical standard in the diagnosis of a genetic disease and in the prediction of short- and long-term morbidity in patients with congenital-heart-disease. At last, I have shown that a neural network- powered strategy for testing and isolating individuals adapts to the parameters of a disease outbreak achieves an epidemic containment.

The interdisciplinary nature of the work in this thesis has allowed the application of new technologies developed in the field of physics to solve problems in the fields of biology and biomedicine, as well as overcoming barriers for the continued revolutionization of deep learning in microscopy.

Thesis: http://hdl.handle.net/2077/67506

Supervisor: Giovanni Volpe
Examiner: Bernhard Mehlig
Opponent: Carolina Wählby
Committee: Marj Tonini, Maria Garcia-Parajo, Alexander Rohrbach

Screenshots from Saga Helgadottir’s PhD Thesis defense.

PhD Thesis presentation: Saga Helgadottir, Giovanni Volpe (Supervisor), Raimund Feifel (GU Physics), Carolina Wählby (Opponent), Marj Tonini (Committee member), Maria Garcia-Parajo (Committee member), Måns Henningson (GU Physics Department Chair), Alexander Rohrbach (Committee member).

PhD Thesis presentation content slide (1).

PhD Thesis presentation content slide (2).

PhD Thesis presentation conclusion slide.

Presentation by L. Natali at Spatial Data Science 2020, 11 June 2021

Comparison of different evolution regimes of disease spreading: free evolution (bottom left half) vs network strategy (top right half). *(Image by Laura Natali.)*

Improving epidemic testing and containment strategies using machine learning.
Laura Natali, Saga Helgadottir, Onofrio M. Maragò, Giovanni Volpe.
Submitted to SDS2020
Date: 11 June
Time: 16:15 (CEST)

Abstract:
Containment of epidemic outbreaks entails great societal and economic costs. Cost-effective containment strategies rely on efficiently identifying infected individuals, making the best possible use of the available testing resources. Therefore, quickly identifying the optimal testing strategy is of critical importance. Here, we demonstrate that machine learning can be used to identify which individuals are most beneficial to test, automatically and dynamically adapting the testing strategy to the characteristics of the disease outbreak. Specifically, we simulate an outbreak using the archetypal susceptible-infectious-recovered (SIR) model and we use data about the first confirmed cases to train a neural network that learns to make predictions about the rest of the population. Using these prediction, we manage to contain the outbreak more effectively and more quickly than with standard approaches. Furthermore, we demonstrate how this method can be used also when there is a possibility of reinfection (SIRS model) to efficiently eradicate an endemic disease.

Improving epidemic testing and containment strategies using machine learning published in Machine Learning: Science and Technology

Comparison of different evolution regimes of disease spreading: free evolution (bottom left half) vs network strategy (top right half).

Improving epidemic testing and containment strategies using machine learning
Laura Natali, Saga Helgadottir, Onofrio M. Maragò, Giovanni Volpe
Machine Learning: Science and Technology, 2 035007 (2021)
doi: 10.1088/2632-2153/abf0f7
arXiv: 2011.11717

Containment of epidemic outbreaks entails great societal and economic costs. Cost-effective containment strategies rely on efficiently identifying infected individuals, making the best possible use of the available testing resources. Therefore, quickly identifying the optimal testing strategy is of critical importance. Here, we demonstrate that machine learning can be used to identify which individuals are most beneficial to test, automatically and dynamically adapting the testing strategy to the characteristics of the disease outbreak. Specifically, we simulate an outbreak using the archetypal susceptible-infectious-recovered (SIR) model and we use data about the first confirmed cases to train a neural network that learns to make predictions about the rest of the population. Using these prediction, we manage to contain the outbreak more effectively and more quickly than with standard approaches. Furthermore, we demonstrate how this method can be used also when there is a possibility of reinfection (SIRS model) to efficiently eradicate an endemic disease.