DeepTrack: A comprehensive deep learning framework for digital microscopy

DeepTrack: A comprehensive deep learning framework for digital microscopy
Benjamin Midtvedt, Saga Helgadottir, Aykut Argun, Daniel Midtvedt, Giovanni Volpe
Click here to see the slides.

Despite the rapid advancement of deep learningmethods for image analysis, they remain under-utilized for the analysis of digital microscopy images. State of the artmethods require expertise in deep learning to implement, disconnecting the development of new methods from end-users. The packages that are available are typically highly specialized, diicult to reappropriate and almost impossible to interface with other methods. Finally, it is prohibitively difficult to procure representative datasets with corresponding labels. DeepTrack is a deep learning framework targeting optical microscopy, designed to account for each of these issues. Firstly, it is packaged with an easy-to-use graphical user interface, solving standard microscopy problems with no required programming experience. Secondly, it provides a comprehensive programming API for creating representative synthetic data, designed to exactly suit the problem. DeepTrack images samples of refractive index or flourophore distributions using physical simulations of customizable optical systems. To accurately represent the data to be analyzed, DeepTrack supports arbitrary optical aberration and experimental noise. Thirdly, many standard deep learning methods are packaged with DeepTrack, including architectures such as U-NET, and regularization techniques such as augmentations. Finally, the framework is fully modular and easily extendable to implement new methods, providing both longevity and a centralized foundation to deploy new deep learning solutions. To demonstrate the versatility of the framework,we show a few typical use-cases, including cell-counting in dense biological samples, extracting 3-dimensional tracks from 2-dimensional videos, and distinguishing and tracking microorganisms in bright-field videos.

Poster Session
Time: June 22nd 2020
Place: Twitter and virtual reality

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Saga Helgadottir – POM Poster – Page 1
Saga Helgadottir – POM Poster – Page 2
Saga Helgadottir – POM Poster – Page 3
Saga Helgadottir – POM Poster – Page 4

Holographic characterisation of subwavelength particles enhanced by deep learning on ArXiv

Phase and amplitude signals from representative particles for testing the performance of the Deep-learning approach

Holographic characterisation of subwavelength particles enhanced by deep learning
Benjamin Midtvedt, Erik Olsén, Fredrik Eklund, Fredrik Höök, Caroline Beck Adiels, Giovanni Volpe, Daniel Midtvedt
arXiv: 2006.11154

The characterisation of the physical properties of nanoparticles in their native environment plays a central role in a wide range of fields, from nanoparticle-enhanced drug delivery to environmental nanopollution assessment. Standard optical approaches require long trajectories of nanoparticles dispersed in a medium with known viscosity to characterise their diffusion constant and, thus, their size. However, often only short trajectories are available, while the medium viscosity is unknown, e.g., in most biomedical applications. In this work, we demonstrate a label-free method to quantify size and refractive index of individual subwavelength particles using two orders of magnitude shorter trajectories than required by standard methods, and without assumptions about the physicochemical properties of the medium. We achieve this by developing a weighted average convolutional neural network to analyse the holographic images of the particles. As a proof of principle, we distinguish and quantify size and refractive index of silica and polystyrene particles without prior knowledge of solute viscosity or refractive index. As an example of an application beyond the state of the art, we demonstrate how this technique can monitor the aggregation of polystyrene nanoparticles, revealing the time-resolved dynamics of the monomer number and fractal dimension of individual subwavelength aggregates. This technique opens new possibilities for nanoparticle characterisation with a broad range of applications from biomedicine to environmental monitoring.

Enhanced force-field calibration via machine learning

Enhanced force-field calibration via machine learning
Aykut Argun, Tobias Thalheim, Stefano Bo, Frank Cichos, Giovanni Volpe

Click here to see the slides.
Twitter Link: here.

The influence of microscopic force fields on the motion of Brownian particles plays a fundamental role in a broad range of fields, including soft matter, biophysics, and active matter. Often, the experimental calibration of these force fields relies on the analysis of the trajectories of these Brownian particles. However, such an analysis is not always straightforward, especially if the underlying force fields are non-conservative or time-varying, driving the system out of thermodynamic equilibrium. Here, we introduce a toolbox to calibrate microscopic force fields by analyzing the trajectories of a Brownian particle using machine learning, namely recurrent neural networks. We demonstrate that this machine-learning approach outperforms standard methods when characterizing the force fields generated by harmonic potentials if the available data are limited. More importantly, it provides a tool to calibrate force fields in situations for which there are no standard methods, such as non-conservative and time-varying force fields. In order to make this method readily available for other users, we provide a Python software package named DeepCalib, which can be easily personalized and optimized for specific applications.

Poster Session
Time: June 22nd 2020
Place: Twitter

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Aykut Argun – POM Poster – Page 1
Aykut Argun – POM Poster – Page 2
Aykut Argun – POM Poster – Page 3
Aykut Argun – POM Poster – Page 4

Holographic characterisation of subwavelength particles enhanced by deep learning

Holographic characterisation of subwavelength particles enhanced by deep learning
Benjamin Midtvedt, Erik Olsen, Fredrick Eklund, Jan Swenson, Fredrik Höök, Caroline Beck Adiels, Giovanni Volpe and Daniel Midtvedt

Click here to see the slides.
Twitter Link: here.

The characterisation of the physical properties of nanoparticles in their native environment plays a central role in a wide range of fields, from nanoparticle-enhanced drug delivery to environmental nanopollution assessment. Standard optical approaches require long trajectories of nanoparticles dispersed in a medium with known viscosity to characterise their diffusion constant and, thus, their size. However, often only short trajectories are available, while the medium viscosity is unknown, e.g., in most biomedical applications.
In this work, we demonstrate a label-free method to quantify size and refractive index of individual subwavelength particles using two orders of magnitude shorter trajectories than required by standard methods, and without assumptions about the physicochemical properties of the medium. We achieve this by developing a weighted average convolutional neural network to analyse the holographic images of the particles. As a proof of principle, we distinguish and quantify size and refractive index of silica and polystyrene particles without prior knowledge of solute viscosity or refractive index. As an example of an application beyond the state of the art, we demonstrate how this technique can monitor the aggregation of polystyrene nanoparticles, revealing the time-resolved dynamics of the monomer number and fractal dimension of individual subwavelength aggregates.
This technique opens new possibilities for nanoparticle characterisation with a broad range of applications from biomedicine to environmental monitoring.

Poster Session
Time: June 22nd 2020
Place: Twitter

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Daniel Midtvedt – POM Poster – Page 1
Daniel Midtvedt – POM Poster – Page 2
Daniel Midtvedt – POM Poster – Page 3
Daniel Midtvedt – POM Poster – Page 4

Optical force field reconstruction using Brownian trajectories

Optical force field reconstruction using Brownian trajectories
Laura Pérez García, Jaime Donlucas Pérez, Giorgio Volpe, Alejandro V. Arzola & Giovanni Volpe

Click here to see the slides.
Twitter Link: here.

Optical tweezers have evolved into sophisticated tools for the measurement and application of nanoscopic forces; its use ranges from mechanobiology to cooling and trapping atoms.
Despite their ever-growing interest, the methods employed to measure optical forces have not changed much in the last 30 years. The key methods measure the potential function, the autocorrelation function (ACF), or the power spectral density (PSD) of an optically trapped particle’s motion. Unfortunately, all these techniques have some drawbacks: they require large amounts of data acquired for long times (potential) or at high frequency (ACF and PSD); they cannot identify non-conservative force-field components; they can only measure the properties of stable equilibrium positions, and they require setting several parameters carefully and expertly [1]. These shortcomings have limited the possibility of measuring nanoscopic forces in many potential applications, such as experiments with non-conservative force fields and out-of-equilibrium conditions.

We have recently introduced a simple, robust, and fast algorithm that permits to reconstruct microscopic force fields from Brownian trajectories, Force Reconstruction via Maximum-likelihood-estimator Analysis — FORMA. FORMA exploits the fact that in the proximity of an equilibrium position, the force field can be approximated by a linear form, and therefore, optimally estimated using a linear maximum-likelihood-estimator. Its key advantages are that FORMA does not require setting analysis parameters, it executes orders-of-magnitude faster than other more standard methods, and it requires ten times fewer data to achieve the same precision and accuracy. Finally, FORMA also permits the characterization of non-conservative force fields and non-stable equilibrium positions in extended force fields [2].

References:

[1] Jones et al. Optical tweezers: Principles and applications. Cambridge, 2015.
[2] L. Pérez García, et al. Nat. Commun. 9, 5166 (2018).

Poster Session
Time: June 22nd 2020
Place: Twitter

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Laura Pérez García – POM Poster – Page 1
Laura Pérez García – POM Poster – Page 2
Laura Pérez García – POM Poster – Page 3
Laura Pérez García – POM Poster – Page 4

Controlling the dynamics of colloidal particles by critical Casimir forces

Controlling the dynamics of colloidal particles by critical Casimir forces
Alessandro Magazzù, Agnese Callegari, Juan Pablo Staforelli, Andrea Gambassi, Siegfried Dietrich and Giovanni Volpe

Click here to see the slides.
Twitter Link: here.

Critical Casimir forces can play an important role for applications in nano-science and nano-technology, owing to their piconewton strength, nanometric action range, fine tunability as a function of temperature, and exquisite dependence on the surface properties of the involved objects. Here, we investigate the effects of critical Casimir forces on the free dynamics of a pair of colloidal particles dispersed in the bulk of a near-critical binary liquid solvent, using blinking optical tweezers. In particular, we measure the time evolution of the distance between the two colloids to determine their relative diffusion and drift velocity. Furthermore, we show how critical Casimir forces change the dynamic properties of this two-colloid system by studying the temperature dependence of the distribution of the so-called first-passage time, i.e., of the time necessary for the particles to reach for the first time a certain separation, starting from an initially assigned one. These data are in good agreement with theoretical results obtained from Monte Carlo simulations and Langevin dynamics.

Poster Session
Time: June 22nd 2020
Place: Twitter

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Alessandro Magazzù – POM Poster – Page 1
Alessandro Magazzù – POM Poster – Page 2
Alessandro Magazzù – POM Poster – Page 3
Alessandro Magazzù – POM Poster – Page 4

Dynamics of an active nanoparticle in an optical trap

Dynamics of an active nanoparticle in an optical trap
Falko Schmidt, Hana Šípová-Jungová, Mikael Käll, Alois Würger, Giovanni Volpe

Click here to see the slides.
Twitter Link: here.

Active matter systems in non-equilibrium conditions have recently gained great interest from many disciplines such as micro and nanomachines and in living organisms. Probing the dynamics of active Brownian particles (ABPs) under confinement such as found in biological systems gives insight into their non-equilibrium processes. Although previous studies [1-4] have shown the effect of confinement on ABPs on the microscale and macroscale investigating dynamics on the nanoscale remains challenging where thermal fluctuations typically prevail. Here, we are investigating experimentally and theoretically a nanoscopic particle in the harmonic potential of an optical trap and driven away from equilibrium by self-induced concentration gradients. We find that a nanoparticle performs fast orbital rotation at finite distance from the trap center and its probability density shifts from a Gaussian to a skewed distribution. Furthermore, we show that by transfer of spin angular momentum from the trapping beam the direction of the particle’s rotation can be controlled. We develop a theoretical model of this system which reveals that the driving mechanism of such fast rotation is the particle’s non-sphericity providing insight for the development of future nanoscopic engines.

References

[1] S. C. Takatori et al., Nat. Comm., 7, 10694 (2016)
[2] O. Dauchot & V. Démery, Phys. Rev. Lett., 122, 068002 (2019)
[3] A. Pototsky & H. Stark, EPL, 98, 5004 (2012)
[4] F. Schmidt et al., Phys. Rev. Lett., 120, 068004 (2018)

Poster Session
Time: June 22nd 2020
Place: Twitter

POM Conference
Link: 
POM
Time: June 25th 2020
Place: Online

Poster Slides

Falko Schmidt – POM Poster – Page 1
Falko Schmidt – POM Poster – Page 2
Falko Schmidt – POM Poster – Page 3
Falko Schmidt – POM Poster – Page 4

Enhanced force-field calibration via machine learning on ArXiv

Calibration of a harmonic potential using a recurrent neural network (RNN)

Enhanced force-field calibration via machine learning
Aykut Argun, Tobias Thalheim, Stefano Bo, Frank Cichos, Giovanni Volpe
arXiv: 2006.08963

The influence of microscopic force fields on the motion of Brownian particles plays a fundamental role in a broad range of fields, including soft matter, biophysics, and active matter. Often, the experimental calibration of these force fields relies on the analysis of the trajectories of these Brownian particles. However, such an analysis is not always straightforward, especially if the underlying force fields are non-conservative or time-varying, driving the system out of thermodynamic equilibrium. Here, we introduce a toolbox to calibrate microscopic force fields by analyzing the trajectories of a Brownian particle using machine learning, namely recurrent neural networks. We demonstrate that this machine-learning approach outperforms standard methods when characterizing the force fields generated by harmonic potentials if the available data are limited. More importantly, it provides a tool to calibrate force fields in situations for which there are no standard methods, such as non-conservative and time-varying force fields. In order to make this method readily available for other users, we provide a Python software package named DeepCalib, which can be easily personalized and optimized for specific applications.

Soft Matter Lab presentations at the Photonics Online Meet-up, 22 June 2020

Six members of the Soft Matter Lab (Aykut Argun, Falko Schmidt, Laura Pérez-Garcia, Saga Helgadottir, Alessandro Magazzù, Daniel Midtvedt) were selected for poster presentations at the Photonics Online Meet-up (POM).

POM is an entirely free virtual conference. It aims to bring together a community of early career and established researchers from universities, industry, and government working in optics and photonics.

The meeting, at its second edition, will be held on June 25th 2020, 9-14.30 Central European Time. The virtual poster session will take place on June 22nd, on Twitter and virtual reality.

The poster contributions being presented are:

Aykut Argun
Enhanced force-field calibration via machine learning
Twitter Link: here.

Falko Schmidt
Dynamics of an active nanoparticle in an optical trap
Twitter Link: here.

Laura Pérez-García
Optical force field reconstruction using Brownian trajectories
Twitter Link: here.

Saga Helgadottir
DeepTrack: A comprehensive deep learning framework for digital microscopy
Twitter Link: here.

Alessandro Magazzù
Controlling the dynamics of colloidal particles by critical Casimir forces
Twitter Link: here.

Daniel Midtvedt
Holographic characterisation of subwavelength particles enhanced by deep learning
Twitter Link: here.

Link: Photonics Online Meet-up (POM)

Seminar by G. Volpe at ICFO, 16 June 2020

Lucky Encounters: From Optical Tweezers to deep Learning
Giovanni Volpe
ICFO Alumni Seminar (Online)
16 June 2020

In this semi-autobiographical talk, I will look back at my career and its evolution. It all started at ICFO with a PhD on optical tweezers in 2008. It then continued with a series of diverse research projects on different fields: active matter, stochastic thermodynamics, neurosciences and, finally, deep learning. I will emphasize how my career has been shaped by lucky encounters. Encounters that have taken me to places and topics I’d never have imagined beforehand. But it all makes sense, in insight.

Date: 16 June 2020
Time: 15:00
Place: Online