Tag: Jesús Pineda

Roadmap on Deep Learning for Microscopy published in Journal of Physics: Photonics

Spatio-temporal spectrum diagram of microscopy techniques and their applications. (Image by the Authors of the manuscript.)
Roadmap on Deep Learning for Microscopy
Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C.D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu, Li-Yu Yu, Sixian You, Yongtao Liu, Maxim A. Ziatdinov, Sergei V. Kalinin, Arlo Sheridan, Uri Manor, Elias Nehme, Ofri Goldenberg, Yoav Shechtman, Henrik K. Moberg, Christoph Langhammer, Barbora Špačková, Saga Helgadottir, Benjamin Midtvedt, Aykut Argun, Tobias Thalheim, Frank Cichos, Stefano Bo, Lars Hubatsch, Jesus Pineda, Carlo Manzo, Harshith Bachimanchi, Erik Selander, Antoni Homs-Corbera, Martin Fränzl, Kevin de Haan, Yair Rivenson, Zofia Korczak, Caroline Beck Adiels, Mite Mijalkov, Dániel Veréb, Yu-Wei Chang, Joana B. Pereira, Damian Matuszewski, Gustaf Kylberg, Ida-Maria Sintorn, Juan C. Caicedo, Beth A Cimini, Muyinatu A. Lediju Bell, Bruno M. Saraiva, Guillaume Jacquemet, Ricardo Henriques, Wei Ouyang, Trang Le, Estibaliz Gómez-de-Mariscal, Daniel Sage, Arrate Muñoz-Barrutia, Ebba Josefson Lindqvist, Johanna Bergman
Journal of Physics: Photonics 8, 012501 (2026)
arXiv: 2303.03793
doi: 10.1088/2515-7647/ae0fd1

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning (ML) are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap encompasses key aspects of how ML is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of ML for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Jesus Pineda defended his PhD thesis on November 11th, 2025. Congrats!

Jesus Pineda defended his PhD thesis on November 11th, 2025. Congrats!
The defense took place in SB-H7 lecture hall, Institutionen för fysik, Johanneberg Campus, Göteborg, at 9:00.

Title: Inductive Biases for Efficient Deep Learning in Microscopy

Abstract: Deep learning has become an indispensable tool for the analysis of microscopy data, yet its integration into routine research remains uneven. Several factors contribute to this gap, including the limited availability of well-annotated datasets and the high computational demands of modern architectures. Microscopy introduces further challenges, as it spans diverse modalities and scales, from proteins to tissues, producing heterogeneous data that defy standardization. Generating reliable annotations also requires expertise and time, while unequal access to high-performance computing further widens the divide between well-resourced institutions and smaller laboratories.

This dissertation argues that the prevailing paradigm of scaling models with ever-larger datasets and computational resources yields diminishing returns for microscopy. Instead, it explores the role of inductive biases as a foundation for building models that are more data-efficient, computationally accessible, and scientifically meaningful. Inductive biases are structural assumptions embedded in model design that guide learning toward patterns aligned with the underlying problem. The first part of this work examines their central role in the advancement of modern deep learning and the diverse ways they shape model behavior.

This potential is demonstrated through three case studies. First, MAGIK employs graph neural networks to analyze biological dynamics in time-lapse microscopy, uncovering local and global properties with high precision, even when trained on limited data. Next, MIRO leverages recurrent graph neural networks to process single-molecule localization datasets, improving the efficiency and reliability of clustering for variable biological structures and scales while retaining strong generalization with minimal supervision. Finally, GAUDI introduces a representation learning framework for characterizing biological systems, providing a physically meaningful representation space for interpretable and transferable analysis.

The findings presented here demonstrate that the integration of inductive biases provides a cohesive strategy to extend the reach of deep learning in the life sciences, enhancing accessibility and ensuring scientific utility under resource constraints.

Thesis: https://gupea.ub.gu.se/items/672c7946-51d6-4773-ad8c-35a3eed41499

Supervisor: Giovanni Volpe
Co-Supervisor: Carlo Manzo
Examiner: Raimund Feifel
Opponent: Anna Kreshuk
Committee: Juliette Griffié, Daniel sage, Daniel Persson
Alternate board member: Jonas Enger

Enhanced spatial clustering of single-molecule localizations with graph neural networks published in Nature Communications

MIRO employs a recurrent graph neural network to refine SMLM point clouds by compressing clusters around their center, enhancing inter-cluster distinction and background separation for efficient clustering. (Image by J. Pineda.)
Enhanced spatial clustering of single-molecule localizations with graph neural networks
Jesús Pineda, Sergi Masó-Orriols, Montse Masoliver, Joan Bertran, Mattias Goksör, Giovanni Volpe and Carlo Manzo
Nature Communications 16, 9693 (2025)
arXiv: 2412.00173
doi: 10.1038/s41467-025-65557-7

Single-molecule localization microscopy generates point clouds corresponding to fluorophore localizations. Spatial cluster identification and analysis of these point clouds are crucial for extracting insights about molecular organization. However, this task becomes challenging in the presence of localization noise, high point density, or complex biological structures. Here, we introduce MIRO (Multifunctional Integration through Relational Optimization), an algorithm that uses recurrent graph neural networks to transform the point clouds in order to improve clustering efficiency when applying conventional clustering techniques. We show that MIRO supports simultaneous processing of clusters of different shapes and at multiple scales, demonstrating improved performance across varied datasets. Our comprehensive evaluation demonstrates MIRO’s transformative potential for single-molecule localization applications, showcasing its capability to revolutionize cluster analysis and provide accurate, reliable details of molecular architecture. In addition, MIRO’s robust clustering capabilities hold promise for applications in various fields such as neuroscience, for the analysis of neural connectivity patterns, and environmental science, for studying spatial distributions of ecological data.

Quantitative evaluation of methods to analyze motion changes in single-particle experiments published on Nature Communications

Rationale for the challenge organization. The interactions of biomolecules in complex environments, such as the cell membrane, regulate physiological processes in living systems. These interactions produce changes in molecular motion that can be used as a proxy to measure interaction parameters. Time-lapse single-molecule imaging allows us to visualize these processes with high spatiotemporal resolution and, in combination with single-particle tracking methods, provide trajectories of individual molecules. (Image by the Authors of the manuscript.)
Quantitative evaluation of methods to analyze motion changes in single-particle experiments
Gorka Muñoz-Gil, Harshith Bachimanchi, Jesús Pineda, Benjamin Midtvedt, Gabriel Fernández-Fernández, Borja Requena, Yusef Ahsini, Solomon Asghar, Jaeyong Bae, Francisco J. Barrantes, Steen W. B. Bender, Clément Cabriel, J. Alberto Conejero, Marc Escoto, Xiaochen Feng, Rasched Haidari, Nikos S. Hatzakis, Zihan Huang, Ignacio Izeddin, Hawoong Jeong, Yuan Jiang, Jacob Kæstel-Hansen, Judith Miné-Hattab, Ran Ni, Junwoo Park, Xiang Qu, Lucas A. Saavedra, Hao Sha, Nataliya Sokolovska, Yongbing Zhang, Giorgio Volpe, Maciej Lewenstein, Ralf Metzler, Diego Krapf, Giovanni Volpe, Carlo Manzo
Nature Communications 16, 6749 (2025)
arXiv: 2311.18100
doi: https://doi.org/10.1038/s41467-025-61949-x

The analysis of live-cell single-molecule imaging experiments can reveal valuable information about the heterogeneity of transport processes and interactions between cell components. These characteristics are seen as motion changes in the particle trajectories. Despite the existence of multiple approaches to carry out this type of analysis, no objective assessment of these methods has been performed so far. Here, we report the results of a competition to characterize and rank the performance of these methods when analyzing the dynamic behavior of single molecules. To run this competition, we implemented a software library that simulates realistic data corresponding to widespread diffusion and interaction models, both in the form of trajectories and videos obtained in typical experimental conditions. The competition constitutes the first assessment of these methods, providing insights into the current limitations of the field, fostering the development of new approaches, and guiding researchers to identify optimal tools for analyzing their experiments.

Global graph features unveiled by unsupervised geometric deep learning on ArXiv

GAUDI leverages a hierarchical graph-convolutional variational autoencoder architecture, where an encoder progressively compresses the graph into a low-dimensional latent space, and a decoder reconstructs the graph from the latent embedding. (Image by M. Granfors and J. Pineda.)
Global graph features unveiled by unsupervised geometric deep learning
Mirja Granfors, Jesús Pineda, Blanca Zufiria Gerbolés, Joana B. Pereira, Carlo Manzo, Giovanni Volpe
arXiv: 2503.05560

Graphs provide a powerful framework for modeling complex systems, but their structural variability makes analysis and classification challenging. To address this, we introduce GAUDI (Graph Autoencoder Uncovering Descriptive Information), a novel unsupervised geometric deep learning framework that captures both local details and global structure. GAUDI employs an innovative hourglass architecture with hierarchical pooling and upsampling layers, linked through skip connections to preserve essential connectivity information throughout the encoding-decoding process. By mapping different realizations of a system – generated from the same underlying parameters – into a continuous, structured latent space, GAUDI disentangles invariant process-level features from stochastic noise. We demonstrate its power across multiple applications, including modeling small-world networks, characterizing protein assemblies from super-resolution microscopy, analyzing collective motion in the Vicsek model, and capturing age-related changes in brain connectivity. This approach not only improves the analysis of complex graphs but also provides new insights into emergent phenomena across diverse scientific domains.

Presentation by J. Pineda at LoG Meetup Sweden, 12 February 2025

MIRO employs a recurrent graph neural network to refine SMLM point clouds by compressing clusters around their center, enhancing inter-cluster distinction and background separation for efficient clustering. (Image by J. Pineda.)
Relational Inductive Biases as a Key to Smarter Deep Learning Microscopy
Jesús Pineda
Learning on graphs and geometry meetup at Uppsala University
Date: 12 February 2025
Time: 11:15
Place: Lecture hall 4101, Ångströmlaboratoriet, Uppsala, Sweden

Geometric deep learning has revolutionized fields like social network analysis, molecular chemistry, and neuroscience, but its application to microscopy data analysis remains a significant challenge. The hurdles stem not only from the scarcity of high-quality data but also from the intrinsic complexity and variability of microscopy datasets. This presentation introduces two groundbreaking geometric deep-learning frameworks designed to overcome these barriers, advancing the integration of graph neural networks (GNNs) into microscopy and unlocking their full potential. First, we present MAGIK, a cutting-edge framework for analyzing biological system dynamics through time-lapse microscopy. Leveraging a graph neural network augmented with attention-based mechanisms, MAGIK processes object features using geometric priors. This enables it to perform a range of tasks, from linking coordinates into trajectories to uncovering local and global dynamic properties with unprecedented precision. Remarkably, MAGIK excels under minimal data conditions, maintaining exceptional performance and robust generalization across diverse scenarios. Next, we introduce MIRO, a novel algorithm powered by recurrent graph neural networks. MIRO pre-processes Single Molecule Localization (SML) datasets to enhance the efficiency of conventional clustering methods. Its ability to handle clusters of varying shapes and scales enables more accurate and consistent analyses across complex datasets. Furthermore, MIRO’s single- and few-shot learning capabilities allow it to generalize effortlessly across scenarios, making it an efficient, scalable, and versatile tool for microscopy data analysis. Together, MAGIK and MIRO address critical limitations in microscopy data analysis, offering innovative solutions for multi-scale data analysis and advancing the boundaries of what is currently achievable with geometric deep learning in the field.

Reference

Pineda, Jesús, Benjamin Midtvedt, Harshith Bachimanchi, Sergio Noé, Daniel Midtvedt, Giovanni Volpe, and Carlo Manzo. Geometric deep learning reveals the spatiotemporal features of microscopic motion. Nat Mach Intell 5, 71–82 (2023).

Pineda, Jesús, Sergi Masó-Orriols, Joan Bertran, Mattias Goksör, Giovanni Volpe, and Carlo Manzo. Spatial Clustering of Molecular Localizations with Graph Neural Networks.  arXiv: 2412.00173 (2024).

Invited talk by J. Pineda at the CMCB Lab on 27 January 2025

MIRO employs a recurrent graph neural network to refine SMLM point clouds by compressing clusters around their center, enhancing inter-cluster distinction and background separation for efficient clustering. (Image by J. Pineda.)
Spatial clustering of molecular localizations with graph neural networks
Jesús Pineda
Date: 27 January 2025
Time: 10:00
Place: SciLifeLab Campus Solna, Sweden

Single-molecule localization microscopy (SMLM) generates point clouds corresponding to fluorophore localizations. Spatial cluster identification and analysis of these point clouds are crucial for extracting insights about molecular organization. However, this task becomes challenging in the presence of localization noise, high point density, or complex biological structures. Here, we introduce MIRO (Multimodal Integration through Relational Optimization), an algorithm that uses recurrent graph neural networks to transform the point clouds in order to improve clustering efficiency when applying conventional clustering techniques. We show that MIRO supports simultaneous processing of clusters of different shapes and at multiple scales, demonstrating improved performance across varied datasets. Our comprehensive evaluation demonstrates MIRO’s transformative potential for single-molecule localization applications, showcasing its capability to revolutionize cluster analysis and provide accurate, reliable details of molecular architecture. In addition, MIRO’s robust clustering capabilities hold promise for applications in various fields such as neuroscience, for the analysis of neural connectivity patterns, and environmental science, for studying spatial distributions of ecological data.

Reference
Pineda, Jesús, Sergi Masó-Orriols, Joan Bertran, Mattias Goksör, Giovanni Volpe, and Carlo Manzo. Spatial Clustering of Molecular Localizations with Graph Neural Networks.  arXiv: 2412.00173

Cross-modality transformations in biological microscopy enabled by deep learning published in Advanced Photonics

Cross-modality transformation and segmentation. (Image by the Authors of the manuscript.)
Cross-modality transformations in biological microscopy enabled by deep learning
Dana Hassan, Jesús Domínguez, Benjamin Midtvedt, Henrik Klein Moberg, Jesús Pineda, Christoph Langhammer, Giovanni Volpe, Antoni Homs Corbera, Caroline B. Adiels
Advanced Photonics 6, 064001 (2024)
doi: 10.1117/1.AP.6.6.064001

Recent advancements in deep learning (DL) have propelled the virtual transformation of microscopy images across optical modalities, enabling unprecedented multimodal imaging analysis hitherto impossible. Despite these strides, the integration of such algorithms into scientists’ daily routines and clinical trials remains limited, largely due to a lack of recognition within their respective fields and the plethora of available transformation methods. To address this, we present a structured overview of cross-modality transformations, encompassing applications, data sets, and implementations, aimed at unifying this evolving field. Our review focuses on DL solutions for two key applications: contrast enhancement of targeted features within images and resolution enhancements. We recognize cross-modality transformations as a valuable resource for biologists seeking a deeper understanding of the field, as well as for technology developers aiming to better grasp sample limitations and potential applications. Notably, they enable high-contrast, high-specificity imaging akin to fluorescence microscopy without the need for laborious, costly, and disruptive physical-staining procedures. In addition, they facilitate the realization of imaging with properties that would typically require costly or complex physical modifications, such as achieving superresolution capabilities. By consolidating the current state of research in this review, we aim to catalyze further investigation and development, ultimately bringing the potential of cross-modality transformations into the hands of researchers and clinicians alike.

Crystallization and topology-induced dynamical heterogeneities in soft granular clusters published in Physical Review of Research

Scheme of the microfluidic system for the production of clusters of a soft granular medium, and Snapshots of the cluster at different times corresponding to different sections of the channel. (Image by the Authors of the manuscript.)
Crystallization and topology-induced dynamical heterogeneities in soft granular clusters
Michal Bogdan, Jesus Pineda, Mihir Durve, Leon Jurkiewicz, Sauro Succi, Giovanni Volpe, Jan Guzowski
Physical Review of Research, 6, L032031 (2024)
DOI: 10.1103/PhysRevResearch.6.L032031
arXiv: 2302.05363

Soft-granular media, such as dense emulsions, foams or tissues, exhibit either fluid- or solidlike properties depending on the applied external stresses. Whereas bulk rheology of such materials has been thoroughly investigated, the internal structural mechanics of finite soft-granular structures with free interfaces is still poorly understood. Here, we report the spontaneous crystallization and melting inside a model soft granular cluster—a densely packed aggregate of N~30-40 droplets engulfed by a fluid film—subject to a varying external flow. We develop machine learning tools to track the internal rearrangements in the quasi-two-dimensional cluster as it transits a sequence of constrictions. As the cluster relaxes from a state of strong mechanical deformations, we find differences in the dynamics of the grains within the interior of the cluster and those at its rim, with the latter experiencing larger deformations and less frequent rearrangements, effectively acting as an elastic membrane around a fluidlike core. We conclude that the observed structural-dynamical heterogeneity results from an interplay of the topological constrains, due to the presence of a closed interface, and the internal solid-fluid transitions. We discuss the universality of such behavior in various types of finite soft granular structures, including biological tissues.

Book “Deep Learning Crash Course” published at No Starch Press

The book Deep Learning Crash Course, authored by Giovanni Volpe, Benjamin Midtvedt, Jesús Pineda, Henrik Klein Moberg, Harshith Bachimanchi, Joana B. Pereira, and Carlo Manzo, has been published online by No Starch Press in July 2024.

Preorder Discount
A preorder discount is available: preorders with coupon code PREORDER will receive 25% off. Link: Preorder @ No Starch Press | Deep Learning Crash Course

Links
@ No Starch Press

Citation 
Giovanni Volpe, Benjamin Midtvedt, Jesús Pineda, Henrik Klein Moberg, Harshith Bachimanchi, Joana B. Pereira, and Carlo Manzo. Deep Learning Crash Course. No Starch Press.
ISBN-13: 9781718503922