DeepTrack 2.1 Logo. (Image from DeepTrack 2.1 Project)How can deep learning enhance microscopy?
Giovanni Volpe SPAOM 2024 Date: 22 November 2024 Time: 10:15-10:45 Place: Toledo, Spain
Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we have introduced a software, currently at version DeepTrack 2.1, to design, train and validate deep-learning solutions for digital microscopy.
The three RMS Early Career Award speakers (l to r) Harshith Bachimanchi, Akaash Kumar and Liam Rooney. (Image by RMS.)Harshith Bachimanchi is shortlisted as one of the RMS (Royal Microscopical Society) early-career award speakers at RMS AMG 2024 (RMS Annual General Meeting 2024) held in London, UK, on 2 October 2024.
In this meeting, Harshith presented his work on leveraging deep learning as a powerful tool to enhance the microscopic data analysis pipelines, to study microorganisms in unprecedented detail. Taking holographic microscopy as an example, he demonstrated that combining holography with deep learning can be used to follow marine micro-organisms through out their lifespan, continuously measuring their three-dimensional positions and dry mass. He also presented some recent results on using deep learning to transform microscopy images from one modality to another (For eg., from Holography to Bright-field and vice versa).
The annual Early Career Award—for which Harshith is shortlisted as one of the potential candidates—recognises the achievements of an outstanding early career imaging scientist in their contribution to microscopy, image analysis, or cytometry.
From RMS:
We heard some fantastic talks earlier from the RMS #EarlyCareer Award speakers at our AGM.
Representation of DNA stretching experiment with the miniTweezer. (Image by A. Ciarlo)miniTweezers2.0: smart optical tweezers for health and life sciences
Antonio Ciarlo Italy-Sweden bilateral workshop on smart sensor technologies and applications Date: 1 October 2024 Time: 14:40-15:05 Place: Meeting Room Kronan, Studenthuset, Linköping University, Campus Valla
Optical tweezers have become indispensable tools in various scientific fields such as biology, physics, chemistry, and materials science. Their wide range of applications has attracted the interest of scientists with limited expertise in optics and physics. Therefore, it is crucial to have a system that is accessible to non-experts. In this study, we present miniTweezers2.0, a highly versatile and user-friendly instrument enhanced by artificial intelligence. We demonstrate the capabilities of the system through three autonomous case study experiments. The first is DNA stretching, a fundamental experiment in single-molecule force spectroscopy. The second experiment focuses on stretching red blood cells, providing insight into their membrane stiffness. The final experiment examines the electrostatic interactions between microparticles in different environments. Our results highlight the potential of automated, versatile optical tweezers to advance our understanding of nanoscale and microscale systems by enabling high-throughput, unbiased measurements. The miniTweezers2.0 system successfully demonstrates the integration of artificial intelligence and automation to make optical tweezers more accessible and versatile, especially for health and life sciences. The adaptability of miniTweezers2.0 underscores its potential as a powerful tool for future scientific exploration across multiple disciplines.
(Image created by G. Volpe with the assistance of DALL·E 2)What is a physicist to do in the age of AI?
Giovanni Volpe Gothenburg Lise Meitner Award 2024 Symposium Date: 27 September 2024 Time: 15:00-15:30 Place: PJ Salen
In recent years, the rapid growth of artificial intelligence, particularly deep learning, has transformed fields from natural sciences to technology. While deep learning is often viewed as a glorified form of curve fitting, its advancement to multi-layered, deep neural networks has resulted in unprecedented performance improvements, often surprising experts. As AI models grow larger and more complex, many wonder whether AI will eventually take over the world and what role remains for physicists and, more broadly, humans.
A critical, yet underappreciated fact is that these AI systems rely heavily on vast amounts of training data, most of which are generated and annotated by humans. This dependency raises an intriguing issue: what happens when human-generated data is no longer available, or when AI begins to train on AI-generated data? The phenomenon of AI poisoning, where the quality of AI outputs declines due to self-referencing, demonstrates the limitations of current AI models. For example, in image recognition tasks, such as those involving the MNIST dataset, AI tends to gravitate towards ‘safe’ or average outputs, diminishing originality and accuracy.
In this context, the unique role of humans becomes clear. Physicists, with their capacity for originality, deep understanding of physical phenomena, and the ability to exploit fundamental symmetries in nature, bring invaluable perspectives to the development of AI. By incorporating physics-informed training architectures and embracing the human drive for meaning and discovery, we can guide the future of AI in truly innovative directions. The message is clear: physicists must remain original, pursue their passions, and continue searching for the hidden laws that govern the world and society.
(Image by A. Argun)Deep Learning for Microscopy
Giovanni Volpe Date: 26 September 2024 Place: ESPCI/Sorbonne, Paris, France
Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we have introduced a software, DeepTrack 2.1, to design, train and validate deep-learning solutions for digital microscopy.
Schematic and brightfield image (inset) of the movement of 16μm diameter micromotor under the illumination of linearly polarized 1064nm laser. (Image by G. Wang.)Light-driven metamachines
Gan Wang, Marcel Rey, Antonio Ciarlo, Mohanmmad Mahdi Shanei, Kunli Xiong, Giuseppe Pesce, Mikael Käll and Giovanni Volpe Date: 5 September 2024 Time: 15:45-16:00
The incorporation of Moore’s law into integrated circuits has spurred opportunities for downsizing traditional mechanical components. Despite advancements in single on-chip motors using electrical, optical, and magnetic drives under ~100 μm, creating complex machines with multiple units remains challenging. Here, we developed a ~10 μm on-chip micromotor using a method compatible with complementary metal oxide semiconductors (CMOS) process. The meta-surface is embedded with the motor to control the incident laser beam direction, enabling momentum exchange with light for movement. The rotation direction and speed are adjustable through the meta-surface, along with the intensity and polarization of applied light. By combining these motors and controlling the configuration, we create complex machines with a size similar to traditional machines below 50um, such as the rotary motion mode of multiple gear coupled gear trains, and the linear motion mode combined with rack and pinion, and combine the micro metal The mirror is introduced into the machine to realize the self-controlled scanning function of the laser in a fixed area.
One exemplar of the HEXBUGS used in the experiment. (Image by the Authors of the manuscript.)Active Matter Experiments with Toy Robots
Angelo Barona Balda, Aykut Argun, Agnese Callegari, Giovanni Volpe
SPIE-OTOM, San Diego, CA, USA, 18 – 22 August 2024 Date: 22 August 2024 Time: 3:00 PM – 3:15 PM Place: Conv. Ctr. Room 6D
Active matter is based on concepts of nonequilibrium thermodynamics applied to the most diverse disciplines. Active Brownian particles, unlike their passive counterparts, self-propel and give rise to complex behaviors distinctive of active matter. As the field is relatively recent, active matter still lacks curricular inclusion. Here, we propose macroscopic experiments using Hexbugs, a commercial toy robot, demonstrating effects peculiar of active systems, such as the setting into motion of passive objects via active particles, the sorting of active particles based on their mobility and chirality. Additionally, we provide a demonstration of Casimir-like attraction between planar objects mediated by active particles.
Trajectory of a hexagonal cluster formed by a transparent particle (blu circle) and six light-absorbing particles (red circles) in a traveling sinusoidal optical pattern, in the absence of thermal noise. The direction of the motion of the optical pattern is given by the arrow. The trajectory’s duration is 30 s. (Image by A. Bergsten.)Chiral active molecules formation via non-reciprocal interactions
Agnese Callegari, Niphredil Klint, John Klint, Alfred Bergsten, Alex Lech, and Giovanni Volpe
SPIE-OTOM, San Diego, CA, USA, 18 – 22 August 2024 Date: 19 August 2024 Time: 5:30 PM – 7:00 PM Place: Conv. Ctr. Exhibit Hall A
In 2019, Schmidt et al. demonstrated light-induced assembly of active colloidal molecules. They used two types of colloidal particles in a water-lutidine mixture: one transparent and one slightly absorbing light. In their experiment, this determined a non-reciprocal interaction between light-absorbing and transparent particles and promoted active molecule formation controlled by light. Beyond experimental details, we here explore the effects of this non-reciprocal interaction solely, showing its role in active molecule formation and self-propulsion. Simulation allows for the study of complex light profiles, enabling precise control over assembly and propulsion properties, relevant for targeted microscopic delivery.
Simplified sketch of the neural network used for the simulations of intracavity optical trapping. (Image by A. Callegari.)Neural networks for intracavity optical trapping
Agnese Callegari, Mathias Samuelsson, Antonio Ciarlo, Giuseppe Pesce, David Bronte Ciriza, Alessandro Magazzù, Onofrio M. Maragò, Antonio Sasso, and Giovanni Volpe
SPIE-OTOM, San Diego, CA, USA, 18 – 22 August 2024 Date: 19 August 2024 Time: 5:30 PM – 7:00 PM Place: Conv. Ctr. Exhibit Hall A
Intracavity optical tweezers have been proven successful for trapping microscopic particles at very low average power intensity – much lower than the one in standard optical tweezers. This feature makes them particularly promising for the study of biological samples. The modelling of such systems, though, requires time-consuming numerical simulations that affect its usability and predictive power. With the help of machine learning, we can overcome the numerical bottleneck – the calculation of optical forces, torques, and losses – and reproduce, in simulation, the results in the literature and generalize to the case of counterpropagating-beams intracavity optical trapping.
Schematic of the scattering of a light ray on a Janus particle. (Image by A. Callegari.)Janus particles in a travelling optical landscape
Agnese Callegari, Giovanni Volpe
SPIE-OTOM, San Diego, CA, USA, 18 – 22 August 2024 Date: 19 August 2024 Time: 5:30 PM – 7:00 PM Place: Conv. Ctr. Exhibit Hall A
Janus particles possess dual properties that makes them very versatile for soft and active matter applications. Modeling their interaction with light, including optical force and torque, presents challenges. We present here a model of spherical, metal-coated Janus particles in the geometric optics approximation. Via an extension of the Optical Tweezers Geometrical Optics (OTGO) toolbox, we calculate optical forces, torques, and absorption. Through numerical simulation, we demonstrate control over Janus particle dynamics in traveling-wave optical landscapes by adjusting speed and periodicity.
