Lucky Encounters: From Optical Tweezers to deep Learning
Giovanni Volpe
ICFO Alumni Seminar (Online)
16 June 2020
In this semi-autobiographical talk, I will look back at my career and its evolution. It all started at ICFO with a PhD on optical tweezers in 2008. It then continued with a series of diverse research projects on different fields: active matter, stochastic thermodynamics, neurosciences and, finally, deep learning. I will emphasize how my career has been shaped by lucky encounters. Encounters that have taken me to places and topics I’d never have imagined beforehand. But it all makes sense, in insight.
Benjamin Midtvedt defended his Master Thesis in Engineering Mathematics and Computer Science at Chalmers University of Technology on 15 June 2020. Congrats!
Screenshot of Benjamin Midtvedt’s Master Thesis defence.Title: DeepTrack: A comprehensive deep learning framework for digital microscopy
Despite the rapid advancement of deep-learning methods for image analysis, they remain underutilized for the analysis of microscopy images. State of the art methods require expertise in deep-learning to implement, disconnecting the development of new methods from end-users. The packages that are available are typically highly specialized, challenging to reappropriate, and almost impossible to interface with other methods. Finally, training deep-learning models often requires large datasets of manually annotated images, making it prohibitively difficult to procure training data that accurately represents the problem.
DeepTrack is a deep-learning framework targeting optical microscopy, designed to account for each of these issues. Firstly, it is packaged with an easy-to-use graphical user interface, solving standard microscopy problems with no required programming experience. Secondly, it bypasses the need for manually annotated experimental data by providing a comprehensive programming API for creating representative synthetic data, designed to exactly suit the problem. DeepTrack creates physical simulations of samples described by refractive index or fluorophore distributions, using fully customizable optical systems. To accurately represent the data to be analyzed, DeepTrack supports arbitrary optical aberration and experimental noise. Thirdly, many standard deep-learning methods are packaged with DeepTrack, including architectures such as U-NET, and regularization techniques such as augmentations, decreasing the barrier to entry. Finally, the framework is fully modular and easily extendable to implement new methods, providing both longevity and a centralized foundation to deploy new deep-learning solutions.
We demonstrate the versatility of DeepTrack by training networks to solve a broad range of common microscopy problems, including particle tracking, cell-counting in dense biological samples, multi-particle 3-dimensional tracking, and cell segmentation and classification.
Master Programme: Engineering Mathematics and Computer Science Supervisor: Giovanni Volpe Examiner: Giovanni Volpe Opponents: Aykut Argun and Saga Helgadóttir
Tobias Sandström and Lars Jansson defended their Master Thesis in Complex Adaptive Systems at Chalmers University of Technology on 15 June 2020. Congrats!
Screenshot of Tobias Sandström and Lars Jansson’s Master Thesis defence.Title: Graph Convolutional Neural Networks for Brain Connectivity Analysis
We explore the strengths and limitations of Graph Convolutional Neural Networks (GCNs) for classification of graph structured data. GCNs differs from regular Artificial Neural Networks (ANNs) in that they operate directly on graph structures by defining convolutional operators in a non-euclidean space. We show that GCNs perform well on graph structured data, where regular ANNs typically fail due to the arbitrary ordering of nodes. Different GCN architectures are examined and compared to simplistic ANNs. Tests are initially performed on simulated data sets with implicit class-dissimilarities in regards to graph structures. We demonstrate that GCNs is vital in accurately classifying the simulated data. Network performance is later evaluated on structured MRI-data, displaying cortical thicknesses for 68 regions in the brain of patients with Alzheimer’s disease and a healthy control group. On the structured MRI-data, both GCNs and regular ANNs are shown to be able classifiers. However, it is crucial for the performance of ANNs that an order of nodes can be imposed on the MRI-data from labeled brain regions.
Supervisors: Jonas Andersson & Alice Deimante Neimantaite, Syntronic AB Examiner: Giovanni Volpe, Department of Physics, University of Gothenburg Opponent: Jonathan Bergqvist
The SAIS workshop is a forum for building the Swedish AI research community and nurture networks across academia and industry. Because of the concern for the COVID-19, the workshop this year is an online conference.
The contributions of Saga Helgadottir will be presented according to the following schedule:
Saga Helgadottir Medical Diagnosis with Machine Learning Date: 17 June 2020 Time: 15:00 CEST
Hillevi Wachtmeister defend her Master Thesis in Physics at Chalmers University of Technology on 11 June 2020. Congrats!
Screenshot of Hillevi Wachtmeister’s Master Thesis defence.Title: Tracking marine micro organisms using deep learning
The goal of this project is to develop a software that can be used to study swimming patterns of marine micro organisms. The software is based on a neural network, which is trained to recognize different types of plankton. The predictions from the network are then used to find the positions of the plankton, and then track their movements.
The project is divided into two parts. First, videos containing only one type of plankton, Lingulodinium polyedra and Alexandrium tamarense respectively, are analyzed. A type of neural network, called U-net, is trained to segment the input images into background and plankton sections. From the segmented images, positions can be obtained and then connected to form a trajectory for each plankton. The drift of the plankton movements is calculated and subtracted from the trajectories, and finally the speed and net displacement is calculated. The results from the single plankton experiments are compared to a previous analysis that was made using the algorithmic method TrackMate.
Secondly, videos containing two types of plankton are analyzed. Two experiments are conducted using Strombidium arenicola and Rhodomonas baltica in the first experiment, and Alexandrium tamarense and Rhodomonas baltica in the second. The segmented images, obtained from the U-net, consists of an additional plankton section for the second type of plankton present in the experiment.
The analysis of the single plankton experiments yields longer and fewer trajectories using the U-net method, compared to the previous TrackMate results. This shows that the TrackMate method is losing plankton at more positions, compared to the U-net method. The U-net method is therefore able to track each plankton for a longer time. The multi-plankton experiments proves the network’s ability to distinguish and track multiple plankton at the same time.
Master programme: MPPHYS – Physics Supervisor: Daniel Midtvedt Examiner: Giovanni Volpe Opponent: Frida Eriksson
Date: 11 June 2020, 9:00 Place: Nexus + Online via Zoom
Bright-field microscopy image of a magnetic chain trapped at the liquid-air interface in a bacterial bath
Anisotropic dynamics of a self-assembled colloidal chain in an active bath
Mehdi Shafiei Aporvari, Mustafa Utkur, Emine Ulku Saritas, Giovanni Volpe & Joakim Stenhammar
Soft Matter, 2020, 16, 5609-5614
doi: https://doi.org/10.1039/D0SM00318B
arXiv: 2002.09961
Anisotropic macromolecules exposed to non-equilibrium (active) noise are very common in biological systems, and an accurate understanding of their anisotropic dynamics is therefore crucial. Here, we experimentally investigate the dynamics of isolated chains assembled from magnetic microparticles at a liquid–air interface and moving in an active bath consisting of motile E. coli bacteria. We investigate both the internal chain dynamics and the anisotropic center-of-mass dynamics through particle tracking. We find that both the internal and center-of-mass dynamics are greatly enhanced compared to the passive case, i.e., a system without bacteria, and that the center-of-mass diffusion coefficient D features a non-monotonic dependence as a function of the chain length. Furthermore, our results show that the relationship between the components of D parallel and perpendicular with respect to the direction of the applied magnetic field is preserved in the active bath compared to the passive case, with a higher diffusion in the parallel direction, in contrast to previous findings in the literature. We argue that this qualitative difference is due to subtle differences in the experimental geometry and conditions and the relative roles played by long-range hydrodynamic interactions and short-range collisions.
Sofia Lundborg defended her Master Thesis in Complex Adaptive Systems at Chalmers University of Technology on 4 June 2020. Congrats!
Screenshot of Sofia Lundborg’s Master Thesis defence.Title: Training Binary Deep Neural Networks Using Knowledge Distillation
Binary networks can be used to speed up inference time and make image analysis possible on less powerful devices. When binarizing a network the accuracy drops.
The thesis aimed to investigate how the accuracy of a binary network can be improved by using knowledge distillation.
Three different knowledge distillation methods were tested for various network types. Additionally, different architectures of a residual block in ResNet were suggested and tested. Test on CIFAR10 showed an 1.5% increase in accuracy when using knowledge distillation and an increase of 1.1% when testing on ImageNet dataset. The results indicate that the suggested knowledge distillation method can improve the accuracy of a binary network. Further testing needs to be done to verify the results, especially longer training. However, there is great potential that knowledge distillation can be used to boost the accuracy of binary networks.
Master programme: MPCAS – Complex Adaptive Systems Supervisor: Giovanni Volpe Supervisors @ Bit Addict: Karl Svensson, Fredrik Ring and Niclas Wikström Examiner: Giovanni Volpe Opponent: Viktor Olsson, Wilhelm Tranheden
Time: June 4, 2020 at 15:00 Place: Online via Zoom
Dennis Kristiansson, Adrian Lundell, Fredrik Meisingseth and David Tonderski defended their Bachelor Thesis at Chalmers University of Technology on 27 May 2020. Congrats!
Title: Deep learning for particle tracking
Abstract: The use of machine learning for classication has in recent years spread into a wide range of disciplines, amongst them the detection of particles for particle tracking on microscopy data. We modified the Python package DeepTrack, which makes use of deep learning to detect particles, creating a package called U-Track. By using a new network architecture based on a U-Net, better performance and higher computational efficiency than DeepTrack was achieved on images with multiple particles. Furthermore, functionality to track detected particles over series of frames was developed. The application of U-Track on experimental data from two-dimensional flow nanometry produced tracks consistent with theory, as well as tracking larger quantities of particles over longer periods of time compared to a digital filter based benchmark algorithm.
Supervisors: Daniel Midtvedt, Department of Physics, University of Gothenburg Examiner: Lena Falk, Department of Physics, University of Gothenburg Opponents: Patrik Wallin, Isak Pettersson, Alexei Orekhov, Anna Wisakanto
Characterisation of Physical Processes from Anomalous Diffusion Data Guest Editors
Miguel A Garcia-March, Maciej Lewenstein, Carlo Manzo, Ralf Metzler, Gorka Muñoz-Gil, Giovanni Volpe
Journal of Physics A: Mathematical and Theoretical
URL: Special Issue on Characterisation of Physical Processes from Anomalous Diffusion Data
In many systems, stochastic transport deviates from the standard laws of Brownian motion. Determining the exponent α characterising anomalous diffusion and identifying the physical origin of this behaviour are crucial steps to understanding the nature of the systems under observation. However, the determination of these properties from the analysis of the measured trajectories is often difficult, especially when these trajectories are short, irregularly sampled, or switching between different behaviours.
Over the last years, several methods have been proposed to quantify anomalous diffusion and the underlying physical process, going beyond the classical calculation of the mean squared displacement. More recently, the advent of machine learning has produced a boost in the methods to quantify anomalous diffusion.
The AnDi challenge aims at bringing together a vibrating and multidisciplinary community of scientists working on this problem. The use of the same reference datasets will allow an unbiased assessment of the performance of methods for characterising anomalous diffusion from single trajectories. This Special Issue will report on these approaches and their performance.
The deadline for submissions will be 30th June 2021 and you can submit manuscripts through ScholarOne Manuscripts. All papers will be refereed according to the usual high standards of the journal.
Giovanni Volpe has been awarded with the ERC Proof of Concept Grant for the research project LUCERO: Smart Optofluidic micromanipulation of Biological Samples.
The grant, consisting of 150k EUR, is meant to commercialize the research project LUCERO, providing an innovative method that combines artificial intelligence and optical tweezers to analyze cells easily and inexpensively.
The current technologies for cell analysis have many limitations: they require access to a large number of cells and considerable expertise. The available methods are also labor-intensive and in some cases the cells are destroyed.
The new method developed in LUCERO simplifies the work and lowers the costs of biomedical research by allowing ordinary standard microscopes, which are already in use in biomedical laboratories, to be used to perform the cell analysis.
The method of LUCERO can be used in several areas, from artificial insemination to forensic medicine. It has potentially a large commercial market.
Giovanni Volpe expects that LUCERO will provide around 20 jobs for university-trained experts and researchers within the next five years.
The project LUCERO has already received initial funding and support from two different organizations (Venture Cup and SPIE). Two doctoral students, Falko Schmidt and Martin B. Mojica, are part of LUCERO’s contributors team.
Links: Press release of the Swedish Research Council: in English, in Swedish.
News on Gothenburg University website: in Swedish.
