Category: Presentation

Presentation by B. Midtvedt at SPIE-ETAI, San Diego, 23 August 2023

LodeSTAR tracks the plankton Noctiluca scintillans. (Image by the Authors of the manuscript.)
Single-shot self-supervised object detection
Benjamin Midtvedt, Jesus Pineda, Fredrik Skärberg, Erik Olsén, Harshith Bachimanchi, Emelie Wesén, Elin Esbjörner, Erik Selander, Fredrik Höök, Daniel Midtvedt, Giovanni Volpe
Date: 23 August 2023
Time: 10:30 AM (PDT)

Object detection is a fundamental task in digital microscopy. Recently, machine-learning approaches have made great strides in overcoming the limitations of more classical approaches. The training of state-of-the-art machine-learning methods almost universally relies on either vast amounts of labeled experimental data or the ability to numerically simulate realistic datasets. However, the data produced by experiments are often challenging to label and cannot be easily reproduced numerically. Here, we propose a novel deep-learning method, named LodeSTAR (Low-shot deep Symmetric Tracking And Regression), that learns to detect small, spatially confined, and largely homogeneous objects that have sufficient contrast to the background with sub-pixel accuracy from a single unlabeled experimental image. This is made possible by exploiting the inherent roto-translational symmetries of the data. We demonstrate that LodeSTAR outperforms traditional methods in terms of accuracy. Furthermore, we analyze challenging experimental data containing densely packed cells or noisy backgrounds. We also exploit additional symmetries to extend the measurable particle properties to the particle’s vertical position by propagating the signal in Fourier space and its polarizability by scaling the signal strength. Thanks to the ability to train deep-learning models with a single unlabeled image, LodeSTAR can accelerate the development of high-quality microscopic analysis pipelines for engineering, biology, and medicine.

Presentation by H. Klein Moberg at SPIE-ETAI, San Diego, 23 August 2023

A convolutional neural network characterizes the properties of very small biomolecules without requiring prior detection. (Image by H. Klein Moberg.)
Deep learning for nanofluidic scattering microscopy
Henrik Klein Moberg
Date: 23 August 2023
Time: 8:15 AM PDT

We show that a custom ResNet-inspired CNN architecture trained on simulated biomolecule trajectories surpasses the performance of standard algorithms in terms of tracking and determining the molecular weight and hydrodynamic radius of biomolecules in the low-kDa regime in optical microscopy. We show that high accuracy and precision is retained even below the 10-kDa regime, constituting approximately an order of magnitude improvement in limit of detection compared to current state-of-the-art, enabling analysis of hitherto elusive species of biomolecules such as cytokines (~5-25 kDa) important for cancer research and the protein hormone insulin (~5.6 kDa), potentially opening up entirely new avenues of biological research.

Keynote presentation by G. Volpe at SPIE-MNM, San Diego, 23 August 2023

Active droploids. (Image taken from Nat. Commun. 12, 6005 (2021).)
Critical fluctuations and critical Casimir forces
Giovanni Volpe
Date: 23 August 2023
Time: 8:00 AM PDT

Critical Casimir forces (CCF) are a powerful tool to control the self-assembly and complex behavior of microscopic and nanoscopic colloids. While CCF were theoretically predicted in 1978, their first direct experimental evidence was provided only in 2008, using total internal reflection microscopy (TIRM). Since then, these forces have been investigated under various conditions, for example, by varying the properties of the involved surfaces or with moving boundaries. In addition, a number of studies of the phase behavior of colloidal dispersions in a critical mixture indicate critical Casimir forces as candidates for tuning the self-assembly of nanostructures and quantum dots, while analogous fluctuation-induced effects have been investigated, for example, at the percolation transition of a chemical sol, in the presence of temperature gradients, and even in granular fluids and active matter. In this presentation, I’ll give an overview of this field with a focus on recent results on the measurement of many-body forces in critical Casimir forces, the realization of micro- and nanoscopic engines powered by critical fluctuations, and the creation of light-controllable colloidal molecules and active droploids.

Presentation by A. Callegari at SPIE-MNM, San Diego, 23 August 2023

An illustration of microscopic gold flakes on surface. (Image by F. Schmidt.)
Tunable critical Casimir forces counteract Casimir–Lifshitz attraction
Falko Schmidt, Agnese Callegari, Abdallah Daddi-Moussa-Ider, Battulga Munkhbat, Ruggero Verre, Timur Shegai, Mikael Käll, Hartmut Löwen, Andrea Gambassi and Giovanni Volpe
SPIE-MNM, San Diego, CA, USA, 20 – 24 August 2023
Date: 23 August 2023

Casimir forces in quantum electrodynamics emerge between microscopic metallic objects because of the confinement of the vacuum electromagnetic fluctuations occurring even at zero temperature. Their generalization at finite temperature and in material media are referred to as Casimir–Lifshitz forces. These forces are typically attractive, leading to the widespread problem of stiction between the metallic parts of micro- and nanodevices. Recently, repulsive Casimir forces have been experimentally realized but their reliance on specialized materials prevents their dynamic control and thus limits their further applicability. Here, we experimentally demonstrate that repulsive critical Casimir forces, which emerge in a critical binary liquid mixture upon approaching the critical temperature, can be used to actively control microscopic and nanoscopic objects with nanometer precision. We demonstrate this by using critical Casimir forces to prevent the stiction caused by the Casimir–Lifshitz forces. We study a microscopic gold flake above a flat gold-coated substrate immersed in a critical mixture. Far from the critical temperature, stiction occurs because of dominant Casimir–Lifshitz forces. Upon approaching the critical temperature, we observe the emergence of repulsive critical Casimir forces that are sufficiently strong to counteract stiction. Our method provides a novel way of controlling the distances of micro- and nanostructures using tunable critical Casimir forces to counteract forces such as the Casimir–Lifshitz force, thereby preventing stiction and device failure. Due to the simplicity of our design the concept can be adapted to already existing MEMS and NEMS by, for example, controlling the temperature via light illumination.

Reference
Falko Schmidt, Agnese Callegari, Abdallah Daddi-Moussa-Ider, Battulga Munkhbat, Ruggero Verre, Timur Shegai, Mikael Käll, Hartmut Löwen, Andrea Gambassi and Giovanni Volpe, Tunable critical Casimir forces counteract Casimir-Lifshitz attraction, Nature Physics 19, 271-278 (2023)

Presentation by H. Bachimanchi at SPIE-ETAI, San Diego, 23 August 2023

Planktons imaged under a holographic microscope. (Illustration by J. Heuschele.)
Decoding microplankton life through holographic microscopy and deep learning
Harshith Bachimanchi
23 August 2023, 8:45 AM PDT

The marine microbial food web plays a central role in the global carbon cycle. However, our mechanistic understanding of the ocean is biased toward its larger constituents, while rates and biomass fluxes in the microbial food web are mainly inferred from indirect measurements and ensemble averages. Yet, resolution at the level of the individual microplankton is required to advance our understanding of the microbial food web. Here, we demonstrate that, by combining holographic microscopy with deep learning, we can follow microplanktons throughout their lifespan, continuously measuring their three-dimensional position and dry mass. The deep-learning algorithms circumvent the computationally intensive processing of holographic data and allow rapid measurements over extended time periods. This permits us to reliably estimate growth rates, both in terms of dry mass increase and cell divisions, as well as to measure trophic interactions between species such as predation events. The individual resolution provides information about selectivity, individual feeding rates, and handling times for individual microplanktons. The method is particularly useful to detail the rates and routes of organic matter transfer in micro-zooplankton, the most important and least known group of primary consumers in the oceans. Studying individual interactions in idealized small systems provides insights that help us understand microbial food webs and ultimately larger-scale processes. We exemplify this by detailed descriptions of micro-zooplankton feeding events, cell divisions, and long-term monitoring of single cells from division to division.

The article related to this presentation can be found at the following link: Microplankton life histories revealed by holographic microscopy and deep learning.

Presentation by L. Natali at SPIE-ETAI, San Diego, 22 August 2023

Robot prototype wirelessly powered in a conductive arena. (Photo by L. Natali.)
Experimental realization of supervised learning in a swarm of autonomous robots
Laura Natali
Date: 22 August 2023
Time: 5:30 PM PDT

Artificial neural networks have limitations compared to biological counterparts as the latter can dynamically change connections and recover from damage. To simplify the study of connections evolving over time, we propose using programmable robots as a swarm to perform supervised learning and autonomously restructure the network. This experimental setup offers a way to study the evolution of connections in a simplified system while addressing the complexity of biological neurons. It has the potential to yield insights into the functioning of biological neural networks while providing a practical application in solving tasks.

Presentation by Y.-W. Chang at SPIE-ETAI, San Diego, 22 August 2023

The proposed method enables accurate synthesis of longitudinal tau pathology. (Image by Y.-W. Chang.)
Synthesizing tau pathology from structural brain imaging using deep learning
Yu-Wei Chang, Giovanni Volpe, Joana B Pereira
Date: 22 August 2023
Time: 10:15 AM PDT

In vivo tau-positron emission tomography (PET) is crucial for determining the stage of Alzheimer’s disease (AD). However, this method is expensive, not widely available, and exposes patients to ionizing radiation, which poses a carcinogenic risk. To address this issue, I’ll present our proposed method, a deep-learning synthesis approach for follow-up tau-PET brain images from baseline tau-PET images using a generative adversarial network (GAN). This technique has the potential to provide valuable insights into the progression of AD, the effectiveness of new treatments, and more accurate diagnosis of the disease.

 

Presentation by A. Callegari at SPIE-ETAI, San Diego, 22 August 2023

Focused rays scattered by an ellipsoidal particles (left). Optical torque along y calculated in the x-y plane using ray scattering with a grid of 1600 rays (up, right) and using a trained neural network (down, right). (Image by the Authors of the manuscript.)
Faster and More Accurate Geometrical-Optics Optical Force Calculation Using Neural Networks
David Bronte Ciriza, Alessandro Magazzù, Agnese Callegari, Gunther Barbosa, Antonio A. R. Neves, Maria Antonia Iatì, Giovanni Volpe, and Onofrio M. Maragò
SPIE-ETAI, San Diego, CA, USA, 20 – 24 August 2023
Date: 22 August 2023

Optical tweezers are an established and versatile tool for the optical trapping and manipulation of microscopic object using light. Modelling the interaction between particles and light, i.e., being able to calculate the optical force and torque the light exerts on the particle, is important to both understand the outcome of experiments and help designing the experimental setup to the obtain a certain outcome. Different modelling approximation and relative calculation techniques are employed depending on the size of the particle and the features of the trapping light. In this work, we will focus on the geometrical optics regime, which hold for particles whose size is significantly larger than the wavelength of the light. In this approximation, optical forces and torques are calculated by discretizing the trapping light beam into a set of rays. Each ray, impinging on the particle, is reflected and refracted multiple times and, in this scattering process, transfers momentum and angular momentum to the particle. However, the choice of the discretization, i.e., which and how many rays we use to represent a beam, sets a trade-off between calculation speed and accuracy. Here, we show that using neural networks allows overcoming this limitation, obtaining not only faster but also more accurate simulations. We demonstrate this using an optically trapped spherical particle for which we obtain an analytical solution to use as ground truth. Then, we exploit our neural networks method to study the dynamics of ellipsoidal particles in a double trap, a system that would be computationally impossible otherwise.

Reference
David Bronte Ciriza, Alessandro Magazzù, Agnese Callegari, Gunther Barbosa, Antonio A. R. Neves, Maria A. Iatì, Giovanni Volpe, Onofrio M. Maragò, Faster and more accurate geometrical-optics optical force calculation using neural networks, ACS Photonics 10, 234–241 (2023)

Poster by H. Zhao at SPIE-ETAI, San Diego, 22 August 2023

Low dose and standard dose PET translation. (Image by H. Zhao.)
High quality PET image synthesis using GAN-transformer
Hang Zhao
Date: 21 August 2023
Time: 5:30 PM PDT

Amyloid-beta positron emission tomography (PET) is used for the diagnosis of Alzheimer’s disease (AD). However, the inherent radiation of radioactive tracers used for PET is potentially harmful to the human body. In this study, we present a deep-learning framework for generating high-quality standard-dose PET brain images from scans that have a simulated reduced injected dose of 12.5% of the standard injected dose, thus reducing radiation exposure without compromising image quality. This novel approach achieves remarkable similarity to full-dose images in both visual and quantitative aspects. Our method offers the potential of enabling safer and more accessible PET imaging for early Alzheimer’s disease detection.

Poster by J. Pineda at SPIE-ETAI, San Diego, 21 August 2023

The proposed method allows for robust detection, segmentation, and tracking of soft granular clusters. (Image by J. Pineda.)
Unveiling the complex dynamics of soft granular materials using deep learning
Jesús Pineda
Date: 21 August 2023
Time: 5:30 PM PDT

Soft granular materials, comprising closely packed grains held together by a thin layer of lubricating fluid, display intricate many-body dynamics resulting in complex flows and rheological behavior, including plasticity and viscoelasticity, memory effects, and avalanches. Despite their widespread presence in nature and industrial applications, the structural mechanics and microscale dynamics of soft granular clusters still need to be better understood, especially those under strong confinement or surrounded by free interfaces. This work aims to bridge the gap in understanding the internal dynamics of finite-sized soft granular media by introducing a deep learning approach to characterize the shapes and movements of deformable grains in the material. We demonstrate the reliability and versatility of the method by studying the dynamics of soft granular clusters that self-organize under external flow in various physically relevant scenarios.