Roadmap on Deep Learning for Microscopy on ArXiv

Spatio-temporal spectrum diagram of microscopy techniques and their applications. (Image by the Authors of the manuscript.)
Roadmap on Deep Learning for Microscopy
Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C.D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu, Li-Yu Yu, Sixian You, Yongtao Liu, Maxim A. Ziatdinov, Sergei V. Kalinin, Arlo Sheridan, Uri Manor, Elias Nehme, Ofri Goldenberg, Yoav Shechtman, Henrik K. Moberg, Christoph Langhammer, Barbora Špačková, Saga Helgadottir, Benjamin Midtvedt, Aykut Argun, Tobias Thalheim, Frank Cichos, Stefano Bo, Lars Hubatsch, Jesus Pineda, Carlo Manzo, Harshith Bachimanchi, Erik Selander, Antoni Homs-Corbera, Martin Fränzl, Kevin de Haan, Yair Rivenson, Zofia Korczak, Caroline Beck Adiels, Mite Mijalkov, Dániel Veréb, Yu-Wei Chang, Joana B. Pereira, Damian Matuszewski, Gustaf Kylberg, Ida-Maria Sintorn, Juan C. Caicedo, Beth A Cimini, Muyinatu A. Lediju Bell, Bruno M. Saraiva, Guillaume Jacquemet, Ricardo Henriques, Wei Ouyang, Trang Le, Estibaliz Gómez-de-Mariscal, Daniel Sage, Arrate Muñoz-Barrutia, Ebba Josefson Lindqvist, Johanna Bergman
arXiv: 2303.03793

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Press release on Objective comparison of methods to decode anomalous diffusion

The article Objective comparison of methods to decode anomalous diffusion has been featured in the News of the University of Gothenburg.

The study, published in Nature Communications and co-written by researchers at the Soft Matter Lab of the Department of Physics at the University of Gothenburg, originates from the AnDi Challenge, a competition co-organised by Giovanni Volpe with researchers from University of Vic – Central University of Catalunya, Institute of Photonic Sciences in Barcelona, University of Potsdam, and Valencia Polytechnic University.

The challenge was held during March–November 2020 and consisted of three main tasks concerning anomalous exponent inference, model classification, and trajectory segmentation. The goal was to provide an objective assessment of the performance of methods to characterise anomalous diffusion from single trajectories.

Here the links to the press releases:
English: A scientific competition led to improved methods for analysing the diffusion of particles.
Swedish: En vetenskaplig tävling ledde till förbättrade metoder för att analysera diffusion av partiklar.

Objective comparison of methods to decode anomalous diffusion published in Nature Communications

An illustration of anomalous diffusion. (Image by Gorka Muñoz-Gil.)
Objective comparison of methods to decode anomalous diffusion
Gorka Muñoz-Gil, Giovanni Volpe, Miguel Angel Garcia-March, Erez Aghion, Aykut Argun, Chang Beom Hong, Tom Bland, Stefano Bo, J. Alberto Conejero, Nicolás Firbas, Òscar Garibo i Orts, Alessia Gentili, Zihan Huang, Jae-Hyung Jeon, Hélène Kabbech, Yeongjin Kim, Patrycja Kowalek, Diego Krapf, Hanna Loch-Olszewska, Michael A. Lomholt, Jean-Baptiste Masson, Philipp G. Meyer, Seongyu Park, Borja Requena, Ihor Smal, Taegeun Song, Janusz Szwabiński, Samudrajit Thapa, Hippolyte Verdier, Giorgio Volpe, Arthur Widera, Maciej Lewenstein, Ralf Metzler, and Carlo Manzo
Nat. Commun. 12, Article number: 6253 (2021)
doi: 10.1038/s41467-021-26320-w
arXiv: 2105.06766

Deviations from Brownian motion leading to anomalous diffusion are found in transport dynamics from quantum physics to life sciences. The characterization of anomalous diffusion from the measurement of an individual trajectory is a challenging task, which traditionally relies on calculating the trajectory mean squared displacement. However, this approach breaks down for cases of practical interest, e.g., short or noisy trajectories, heterogeneous behaviour, or non-ergodic processes. Recently, several new approaches have been proposed, mostly building on the ongoing machine-learning revolution. To perform an objective comparison of methods, we gathered the community and organized an open competition, the Anomalous Diffusion challenge (AnDi). Participating teams applied their algorithms to a commonly-defined dataset including diverse conditions. Although no single method performed best across all scenarios, machine-learning-based approaches achieved superior performance for all tasks. The discussion of the challenge results provides practical advice for users and a benchmark for developers.

Classification, inference and segmentation of anomalous diffusion with recurrent neural networks published in Journal of Physics A: Mathematical and Theoretical

RANDI architecture to classify the model underlying anomalous diffusion.
Classification, inference and segmentation of anomalous diffusion with recurrent neural networks
Aykut Argun, Giovanni Volpe, Stefano Bo
J. Phys. A: Math. Theor. 54 294003 (2021)
doi: 10.1088/1751-8121/ac070a
arXiv: 2104.00553

Countless systems in biology, physics, and finance undergo diffusive dynamics. Many of these systems, including biomolecules inside cells, active matter systems and foraging animals, exhibit anomalous dynamics where the growth of the mean squared displacement with time follows a power law with an exponent that deviates from 1. When studying time series recording the evolution of these systems, it is crucial to precisely measure the anomalous exponent and confidently identify the mechanisms responsible for anomalous diffusion. These tasks can be overwhelmingly difficult when only few short trajectories are available, a situation that is common in the study of non-equilibrium and living systems. Here, we present a data-driven method to analyze single anomalous diffusion trajectories employing recurrent neural networks, which we name RANDI. We show that our method can successfully infer the anomalous exponent, identify the type of anomalous diffusion process, and segment the trajectories of systems switching between different behaviors. We benchmark our performance against the state-of-the art techniques for the study of single short trajectories that participated in the Anomalous Diffusion (AnDi) challenge. Our method proved to be the most versatile method, being the only one to consistently rank in the top 3 for all tasks proposed in the AnDi challenge.

Aykut Argun’s team wins in four categories of the ANDI challenge

Aykut Argun (Soft Matter Lab) and Stefano Bo (MPI Dresden) participated in the AnDi Challenge, the Anomalous Diffusion challenge, in all the nine categories.

The challenge consisted of different tasks, specifically:

  • Task 1 – Inference of the anomalous diffusion exponent α.
  • Task 2 – Classification of the diffusion model.
  • Task 3 – Segmentation of trajectories.

Each task included modalities for different number of dimensions (1D, 2D and 3D), for a total of 9 subtasks.

Approximately 20 teams from all the world participated in the challenge.

Aykut’s and Stefano’s team, eduN, ranked in the first three positions in all the categories. EduN won the 1st place in 4 of the categories, i.e., Task 2 (1D and 2D), and Task 3 (1D and 3D), the 2nd place in another 4 categories, and 3rd in the remaining category.

The details and the information about the final results can be found on ANDI Challenge final results page: http://www.andi-challenge.org/ (select: Learn the Details and then Final Results)

Here the link to the video of the announcement.

Enhanced force-field calibration via machine learning featured in AIP SciLight

The article Enhanced force-field calibration via machine learning
has been featured in: “Machine Learning Outperforms Standard Force-Field Calibration Techniques”, AIP SciLight (November 6, 2020).

Scilight showcases the most interesting research across the physical sciences published in AIP Publishing Journals.

Scilight is published weekly (52 issues per year) by AIP Publishing.

Enhanced force-field calibration via machine learning published in Applied Physics Reviews

Representation of a particle in a force field
Enhanced force-field calibration via machine learning
Aykut Argun, Tobias Thalheim, Stefano Bo, Frank Cichos, Giovanni Volpe
Applied Physics Reviews 7, 041404 (2020)
doi: 10.1063/5.0019105
arXiv: 2006.08963

The influence of microscopic force fields on the motion of Brownian particles plays a fundamental role in a broad range of fields, including soft matter, biophysics, and active matter. Often, the experimental calibration of these force fields relies on the analysis of the trajectories of these Brownian particles. However, such an analysis is not always straightforward, especially if the underlying force fields are non-conservative or time-varying, driving the system out of thermodynamic equilibrium. Here, we introduce a toolbox to calibrate microscopic force fields by analyzing the trajectories of a Brownian particle using machine learning, namely recurrent neural networks. We demonstrate that this machine-learning approach outperforms standard methods when characterizing the force fields generated by harmonic potentials if the available data are limited. More importantly, it provides a tool to calibrate force fields in situations for which there are no standard methods, such as non-conservative and time-varying force fields. In order to make this method readily available for other users, we provide a Python software package named DeepCalib, which can be easily personalized and optimized for specific applications.

Funding:

ERC-founder H2020 European Research Council (ERC) Starting Grant ComplexSwimmers (677511).

Anomalous Diffusion Measurement with Neural Networks published in Phys Rev E

Measurement of Anomalous Diffusion Using Recurrent Neural Networks

Measurement of Anomalous Diffusion Using Recurrent Neural Networks
Stefano Bo, Falko Schmidt, Ralf Eichborn & Giovanni Volpe
Physical Review E 100(1), 010102(R) (2019)
doi: 10.1103/PhysRevE.100.010102
arXiv: 1905.02038

Anomalous diffusion occurs in many physical and biological phenomena, when the growth of the mean squared displacement (MSD) with time has an exponent different from one. We show that recurrent neural networks (RNN) can efficiently characterize anomalous diffusion by determining the exponent from a single short trajectory, outperforming the standard estimation based on the MSD when the available data points are limited, as is often the case in experiments. Furthermore, the RNN can handle more complex tasks where there are no standard approaches, such as determining the anomalous diffusion exponent from a trajectory sampled at irregular times, and estimating the switching time and anomalous diffusion exponents of an intermittent system that switches between different kinds of anomalous diffusion. We validate our method on experimental data obtained from sub-diffusive colloids trapped in speckle light fields and super-diffusive microswimmers.

Minimal Microscopic Heat Engine published in Phys. Rev. E

Experimental realization of a minimal microscopic heat engine

Experimental realization of a minimal microscopic heat engine
Aykut Argun, Jalpa Soni, Lennart Dabelow, Stefano Bo, Giuseppe Pesce, Ralf Eichhorn & Giovanni Volpe
Physical Review E 96(5), 052106 (2017)
DOI: 10.1103/PhysRevE.96.052106
arXiv: 1708.07197

Microscopic heat engines are microscale systems that convert energy flows between heat reservoirs into work or systematic motion. We have experimentally realized a minimal microscopic heat engine. It consists of a colloidal Brownian particle optically trapped in an elliptical potential well and simultaneously coupled to two heat baths at different temperatures acting along perpendicular directions. For a generic arrangement of the principal directions of the baths and the potential, the symmetry of the system is broken, such that the heat flow drives a systematic gyrating motion of the particle around the potential minimum. Using the experimentally measured trajectories, we quantify the gyrating motion of the particle, the resulting torque that it exerts on the potential, and the associated heat flow between the heat baths. We find excellent agreement between the experimental results and the theoretical predictions.