Seminar by D. Midtvedt at Freie Universität Berlin, 29 October 2021

DeepTrack 2.0 Logo. (Image from DeepTrack 2.0 Project)
Quantitative digital microscopy enhanced by deep learning
Daniel Midtvedt
(online at) Freie Universität Berlin, Germany
29 October 2021

Video microscopy has a long history of providing insight and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time-consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis.
However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we recently introduced a software, DeepTrack 2.0, to design, train, and validate deep-learning solutions for digital microscopy.
In this talk, I will show how this software can be used in a broad range of applications, from particle localization, tracking, and characterization, to cell counting and classification. Thanks to its user-friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and thanks to its open-source, object-oriented programing, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience.

Objective comparison of methods to decode anomalous diffusion published in Nature Communications

An illustration of anomalous diffusion. (Image by Gorka Muñoz-Gil.)
Objective comparison of methods to decode anomalous diffusion
Gorka Muñoz-Gil, Giovanni Volpe, Miguel Angel Garcia-March, Erez Aghion, Aykut Argun, Chang Beom Hong, Tom Bland, Stefano Bo, J. Alberto Conejero, Nicolás Firbas, Òscar Garibo i Orts, Alessia Gentili, Zihan Huang, Jae-Hyung Jeon, Hélène Kabbech, Yeongjin Kim, Patrycja Kowalek, Diego Krapf, Hanna Loch-Olszewska, Michael A. Lomholt, Jean-Baptiste Masson, Philipp G. Meyer, Seongyu Park, Borja Requena, Ihor Smal, Taegeun Song, Janusz Szwabiński, Samudrajit Thapa, Hippolyte Verdier, Giorgio Volpe, Arthur Widera, Maciej Lewenstein, Ralf Metzler, and Carlo Manzo
Nat. Commun. 12, Article number: 6253 (2021)
doi: 10.1038/s41467-021-26320-w
arXiv: 2105.06766

Deviations from Brownian motion leading to anomalous diffusion are found in transport dynamics from quantum physics to life sciences. The characterization of anomalous diffusion from the measurement of an individual trajectory is a challenging task, which traditionally relies on calculating the trajectory mean squared displacement. However, this approach breaks down for cases of practical interest, e.g., short or noisy trajectories, heterogeneous behaviour, or non-ergodic processes. Recently, several new approaches have been proposed, mostly building on the ongoing machine-learning revolution. To perform an objective comparison of methods, we gathered the community and organized an open competition, the Anomalous Diffusion challenge (AnDi). Participating teams applied their algorithms to a commonly-defined dataset including diverse conditions. Although no single method performed best across all scenarios, machine-learning-based approaches achieved superior performance for all tasks. The discussion of the challenge results provides practical advice for users and a benchmark for developers.

Invited Talk by G. Volpe at Microscopies and Spectroscopies: Accessing the Nanoscale, 28 October 2021

DeepTrack 2.0 Logo. (Image from DeepTrack 2.0 Project)
Quantitative Digital Microscopy with Deep Learning
Giovanni Volpe
Invited Talk at the XXXVI Trobades Cientifíques de la Mediterránia – Josep Miquel Vidal
Microscopies and Spectroscopies: Accessing the Nanoscale
Menorca, Spain
28 October 2021
11:40 AM

Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we introduce a software, DeepTrack 2.0, to design, train and validate deep- learning solutions for digital microscopy. We use it to exemplify how deep learning can be employed for a broad range of applications, from particle localization, tracking and characterization to cell counting and classification. Thanks to its user- friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and, thanks to its open-source object-oriented programming, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience.

Press release on Active Droploids

The article Active Droploids has been featured in a press release of the University of Gothenburg.

The study, published in Nature Communications, examines a special system of colloidal particles and demonstrates a new kind of active matter, which interacts with and modifies its environment. In the long run, the result of the study can be used for drug delivery inside the human body or to perform sensing of environmental pollutants and their clean-up.

Here the links to the press releases:
English: Feedback creates a new class of active biomimetic materials.
Swedish: Feedback möjliggör en ny form av aktiva biomimetiska material.

The article has been features also in Mirage News, Science Daily, Phys.org, Innovations Report, Informationsdienst Wissenschaft (idw) online, Nanowerk.

Presentation by S. Olsson, 20 October 2021

Machine Learning for Molecular dynamics — Why bother?
Simon Olsson
Chalmers University of Technology
20 October 2021
Online

With faster compute-infrastructures, molecular simulations play an increasingly important role in the basic sciences and application areas such as drug and materials design. Simultaneously, machine learning and artificial intelligence are receiving increased attention due to increasing volumes of data generated both inside and outside of science. In this talk, I will talk about a few applications of these technologies in molecular simulation, focusing on biomolecular simulations [1,2]

[1] Olsson & Noé ”Dynamic Graphical Models of Molecular Kinetics” Proc. Natl. Acad. Sci. U.S.A. (2019) doi: 10.1073/pnas.1901692116.
[2] Noe†, Olsson, Köhler, Wu ”Boltzmann Generators: Sampling Equilibrium States of Many-Body Systems with Deep Learning” Science (2019). 365, eaaw1147. doi:10.1126/science.aaw1147.

Link: http://www.cse.chalmers.se/~simonols/

Keynote Talk by G. Volpe at CIIBBI, 15 October 2021

DeepTrack 2.0 Logo. (Image from DeepTrack 2.0 Project)
Deep Learning for Microscopy with Biomedical Applications
Giovanni Volpe
Keynote Talk at the 2nd International Congress of Biomedical Engineering and Bioengineering
Online
15 October 2021
14:00 CEST

Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we introduce a software, DeepTrack 2.0, to design, train and validate deep-learning solutions for digital microscopy. We use it to exemplify how deep learning can be employed for a broad range of applications, from particle localization, tracking and characterization to cell counting and classification. Thanks to its user-friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and, thanks to its open-source object-oriented programming, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience.

Invited Talk by G. Volpe at 106 RAFA, 15 October 2021

Deep learning for microscopy, optical trapping, and active matter
Giovanni Volpe
Invited Talk for 106 RAFA – Reunión de la Asociación de Física de Argentina – División Materia Blanda
15 October 2021
12:00 PM

After a brief overview of artificial intelligence, machine learning and deep learning, I will present a series of recent works in which we have employed deep learning for applications in photonics and active matter. In particular, I will explain how we employed deep learning to enhance digital video microscopy, to estimate the properties of anomalous diffusion, to characterize microscopic force fields, to improve the calculation of optical forces, and to characterize nanoparticles. Finally, I will provide an outlook for the application of deep learning in photonics and active matter.

Active droploids published in Nature Communications

Active droploids. (Image taken from the article.)
Active droploids
Jens Grauer, Falko Schmidt, Jesús Pineda, Benjamin Midtvedt, Hartmut Löwen, Giovanni Volpe & Benno Liebchen
Nat. Commun. 12, 6005 (2021)
doi: 10.1038/s41467-021-26319-3
arXiv: 2109.10677

Active matter comprises self-driven units, such as bacteria and synthetic microswimmers, that can spontaneously form complex patterns and assemble into functional microdevices. These processes are possible thanks to the out-of-equilibrium nature of active-matter systems, fueled by a one-way free-energy flow from the environment into the system. Here, we take the next step in the evolution of active matter by realizing a two-way coupling between active particles and their environment, where active particles act back on the environment giving rise to the formation of superstructures. In experiments and simulations we observe that, under light-illumination, colloidal particles and their near-critical environment create mutually-coupled co-evolving structures. These structures unify in the form of active superstructures featuring a droplet shape and a colloidal engine inducing self-propulsion. We call them active droploids—a portmanteau of droplet and colloids. Our results provide a pathway to create active superstructures through environmental feedback.

Press release on Extracting quantitative biological information from bright-field cell images using deep learning

Virtually-stained generated image for lipid-droplet.

The article Extracting quantitative biological information from bright-field cell images using deep learning has been featured in a press release of the University of Gothenburg.

The study, recently published in Biophysics Reviews, shows how artificial intelligence can be used to develop faster, cheaper and more reliable information about cells, while also eliminating the disadvantages from using chemicals in the process.

Here the links to the press releases on Cision:
Swedish: Effektivare studier av celler med ny AI-metod
English: More effective cell studies using new AI method

Here the links to the press releases in the News of the University of Gothenburg:
Swedish: Effektivare studier av celler med ny AI-metod
English: More effective cell studies using new AI method

Invited Talk by G. Volpe at Venice meeting on Fluctuations in small complex systems V, 7 October 2021

RANDI architecture to classify the model underlying anomalous diffusion.
Measuring Anomalous Diffusion with Deep Learning
Giovanni Volpe
Invited Talk at Venice meeting on Fluctuations in small complex systems V
Palazzo Franchetti, Venezia, Italy
7 October 2021, 16:30 CEST

Countless systems in biology, physics, and finance undergo diffusive dynamics. Many of these systems, including biomolecules inside cells, active matter systems and foraging animals, exhibit anomalous dynamics where the growth of the mean squared displacement with time follows a power law with an exponent that deviates from 1. When studying time series recording the evolution of these systems, it is crucial to precisely measure the anomalous exponent and confidently identify the mechanisms responsible for anomalous diffusion. These tasks can be overwhelmingly difficult when only few short trajectories are available, a situation that is common in the study of non-equilibrium and living systems. Here, we present a data-driven method to analyze single anomalous diffusion trajectories employing recurrent neural networks, which we name RANDI. We show that our method can successfully infer the anomalous exponent, identify the type of anomalous diffusion process, and segment the trajectories of systems switching between different behaviors. We benchmark our performance against the state-of-the art techniques for the study of single short trajectories that participated in the Anomalous Diffusion (AnDi) challenge. Our method proved to be the most versatile method, being the only one to consistently rank in the top 3 for all tasks proposed in the AnDi challenge.