Fluorescence lifetime imaging microscopy (FLIM) provides detailed information about molecular interactions and biological processes. A major bottleneck for FLIM is image resolution at high acquisition speeds due to the engineering and signal-processing limitations of time-resolved imaging technology. Here, we present single-sample image-fusion upsampling, a data-fusion approach to computational FLIM super-resolution that combines measurements from a low-resolution time-resolved detector (that measures photon arrival time) and a high-resolution camera (that measures intensity only). To solve this otherwise ill-posed inverse retrieval problem, we introduce statistically informed priors that encode local and global correlations between the two “single-sample” measurements. This bypasses the risk of out-of-distribution hallucination as in traditional data-driven approaches and delivers enhanced images compared, for example, to standard bilinear interpolation. The general approach laid out by single-sample image-fusion upsampling can be applied to other image super-resolution problems where two different datasets are available. A computational approach to microscopy enables upsampling fluorescence lifetime images by factors as high as 16x16.
@article{kapitany2024single,title={Single-sample image-fusion upsampling of fluorescence lifetime images},author={Kapit{\'a}ny, Valentin and Fatima, Areeba and Zickus, Vytautas and Whitelaw, Jamie and McGhee, Ewan and Insall, Robert and Machesky, Laura and Faccio, Daniele},journal={Science Advances},volume={10},number={21},pages={eadn0139},year={2024},doi={10.1126/sciadv.adn0139},url={https://www.science.org/doi/abs/10.1126/sciadv.adn0139},eprint={https://www.science.org/doi/pdf/10.1126/sciadv.adn0139},}
2023
AI for time-resolved imaging: from fluorescence lifetime to single-pixel time of flight
@phdthesis{kapitany2023ai,title={AI for time-resolved imaging: from fluorescence lifetime to single-pixel time of flight},author={Kapitány, Valentin},year={2023},school={University of Glasgow},}
@article{kapitany2023single,title={Single-shot time-folded fluorescence lifetime imaging},author={Kapitany, Valentin and Zickus, Vytautas and Fatima, Areeba and Carles, Guillem and Faccio, Daniele},journal={Proceedings of the National Academy of Sciences},volume={120},number={16},pages={e2214617120},year={2023},publisher={National Acad Sciences},}
mmSense: Detecting Concealed Weapons with a Miniature Radar Sensor
Kevin Mitchell , Khaled Kassem , Chaitanya Kaul , Valentin Kapitany , Philip Binner , Andrew Ramsay , and 2 more authors
In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023
@inproceedings{mitchell2023mmsense,title={mmSense: Detecting Concealed Weapons with a Miniature Radar Sensor},author={Mitchell, Kevin and Kassem, Khaled and Kaul, Chaitanya and Kapitany, Valentin and Binner, Philip and Ramsay, Andrew and Faccio, Daniele and Murray-Smith, Roderick},booktitle={ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},pages={1--5},year={2023},organization={IEEE},}
Remote Sensing of Neurodegeneration by Diffuse Correlation Spectroscopy
Philip Binner , Ilya Starshynov , Aisling McFall , Gonzalo Tejeda , Valentin Kapitany , Colin Molloy , and 2 more authors
In Computational Optical Sensing and Imaging , 2023
@inproceedings{binner2023remote,title={Remote Sensing of Neurodegeneration by Diffuse Correlation Spectroscopy},author={Binner, Philip and Starshynov, Ilya and McFall, Aisling and Tejeda, Gonzalo and Kapitany, Valentin and Molloy, Colin and Tobin, Andrew B and Faccio, Daniele},booktitle={Computational Optical Sensing and Imaging},pages={CW5B--6},year={2023},organization={Optica Publishing Group},}
2022
Iscience
Working at the interface of physics and biology: An early career researcher perspective
Claire Dilliway , Oliver Dyer , Elena Mandrou , Daniel Mitchell , Govind Menon , Hugh Sparks , and 2 more authors
@article{dilliway2022working,title={Working at the interface of physics and biology: An early career researcher perspective},author={Dilliway, Claire and Dyer, Oliver and Mandrou, Elena and Mitchell, Daniel and Menon, Govind and Sparks, Hugh and Kapitany, Valentin and Payne-Dwyer, Alex},journal={Iscience},volume={25},number={12},year={2022},publisher={Elsevier}}
2021
3D imaging from multipath temporal echoes
Alex Turpin , Valentin Kapitany , Jack Radford , Davide Rovelli , Kevin Mitchell , Ashley Lyons , and 2 more authors
@article{turpin20213d,title={3D imaging from multipath temporal echoes},author={Turpin, Alex and Kapitany, Valentin and Radford, Jack and Rovelli, Davide and Mitchell, Kevin and Lyons, Ashley and Starshynov, Ilya and Faccio, Daniele},journal={Physical Review Letters},volume={126},number={17},pages={174301},year={2021},publisher={APS},}
SPIE
Megapixel time-gated SPAD image sensor for scientific imaging applications
Kazuhiro Morimoto , Andrei Ardelean , Ming-Lo Wu , Arin Can Ulku , Ivan Michel M Antolovic , Vytautas Zickus , and 5 more authors
In High-Speed Biomedical Imaging and Spectroscopy VI , 2021
@inproceedings{morimoto2021megapixel,title={Megapixel time-gated SPAD image sensor for scientific imaging applications},author={Morimoto, Kazuhiro and Ardelean, Andrei and Wu, Ming-Lo and Ulku, Arin Can and Antolovic, Ivan Michel M and Zickus, Vytautas and Kapitany, Valentin and Fatima, Areeba and Turpin, Alex and Insall, Robert and others},booktitle={High-Speed Biomedical Imaging and Spectroscopy VI},volume={11654},pages={116540U},year={2021},organization={SPIE},}
Optica
Multipath temporal echoes for reconstructing spatial scenes
Valentin Kapitany , Alex Turpin , Jack Radford , Davide Rovelli , Kevin Mitchell , Ashley Lyons , and 2 more authors
In Computational Optical Sensing and Imaging , 2021
@inproceedings{kapitany2021multipath,title={Multipath temporal echoes for reconstructing spatial scenes},author={Kapitany, Valentin and Turpin, Alex and Radford, Jack and Rovelli, Davide and Mitchell, Kevin and Lyons, Ashley and Starshynov, Ilya and Faccio, Daniele},booktitle={Computational Optical Sensing and Imaging},pages={CW5B--3},year={2021},organization={Optica Publishing Group},}
COSI
Multipath temporal echoes for reconstructing spatial scenes
Valentin Kapitany , Alex Turpin , Jack Radford , Davide Rovelli , Kevin Mitchell , Ashley Lyons , and 2 more authors
In Computational Optical Sensing and Imaging , 2021
@inproceedings{kapitany2021multipati,title={Multipath temporal echoes for reconstructing spatial scenes},author={Kapitany, Valentin and Turpin, Alex and Radford, Jack and Rovelli, Davide and Mitchell, Kevin and Lyons, Ashley and Starshynov, Ilya and Faccio, Daniele},booktitle={Computational Optical Sensing and Imaging},pages={CW5B--3},year={2021},organization={Optica Publishing Group},}
CLEO Invited Talk
Inferring spatial scenes from their time-resolved multipath echoes
Valentin Kapitany , Alex Turpin , Jack Radford , Davide Rovelli , Ashley Lyons , Ilya Starshynov , and 1 more author
In 2021 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC) , 2021
@inproceedings{kapitany2021inferring,award={Invited Talk},title={Inferring spatial scenes from their time-resolved multipath echoes},author={Kapitany, Valentin and Turpin, Alex and Radford, Jack and Rovelli, Davide and Lyons, Ashley and Starshynov, Ilya and Faccio, Daniele},booktitle={2021 Conference on Lasers and Electro-Optics Europe \& European Quantum Electronics Conference (CLEO/Europe-EQEC)},pages={1--1},year={2021},organization={IEEE},}
2020
Sci. Rep.
Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation
Vytautas Zickus , Ming-Lo Wu , Kazuhiro Morimoto , Valentin Kapitany , Areeba Fatima , Alex Turpin , and 5 more authors
@article{zickus2020fluorescence,title={Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation},author={Zickus, Vytautas and Wu, Ming-Lo and Morimoto, Kazuhiro and Kapitany, Valentin and Fatima, Areeba and Turpin, Alex and Insall, Robert and Whitelaw, Jamie and Machesky, Laura and Bruschini, Claudio and others},journal={Scientific Reports},volume={10},number={1},pages={20986},year={2020},publisher={Nature Publishing Group UK London}}
Optica
Spatial images from temporal data
Alex Turpin , Gabriella Musarra , Valentin Kapitany , Francesco Tonolini , Ashley Lyons , Ilya Starshynov , and 5 more authors
@article{turpin2020spatial,title={Spatial images from temporal data},author={Turpin, Alex and Musarra, Gabriella and Kapitany, Valentin and Tonolini, Francesco and Lyons, Ashley and Starshynov, Ilya and Villa, Federica and Conca, Enrico and Fioranelli, Francesco and Murray-Smith, Roderick and others},journal={Optica},volume={7},number={8},pages={900--905},year={2020},publisher={Optica Publishing Group}}
SPIE
Imaging from temporal data via spiking convolutional neural networks
Paul Kirkland , Valentin Kapitany , Ashley Lyons , John Soraghan , Alex Turpin , Daniele Faccio , and 1 more author
In Emerging Imaging and Sensing Technologies for Security and Defence V; and Advanced Manufacturing Technologies for Micro-and Nanosystems in Security and Defence III , 2020
@inproceedings{kirkland2020imaging,title={Imaging from temporal data via spiking convolutional neural networks},author={Kirkland, Paul and Kapitany, Valentin and Lyons, Ashley and Soraghan, John and Turpin, Alex and Faccio, Daniele and Di Caterina, Gaetano},booktitle={Emerging Imaging and Sensing Technologies for Security and Defence V; and Advanced Manufacturing Technologies for Micro-and Nanosystems in Security and Defence III},volume={11540},pages={66--85},year={2020},organization={SPIE},}
COSI
Data fusion for high resolution fluorescence lifetime imaging using deep learning.
Valentin Kapitany , Alex Turpin , Jamie Whitelaw , Ewan McGhee , Robert Insali , Laura Machesky , and 1 more author
In Computational Optical Sensing and Imaging , 2020
@inproceedings{kapitany2020data,title={Data fusion for high resolution fluorescence lifetime imaging using deep learning.},author={Kapitany, Valentin and Turpin, Alex and Whitelaw, Jamie and McGhee, Ewan and Insali, Robert and Machesky, Laura and Faccio, Daniele},booktitle={Computational Optical Sensing and Imaging},pages={CW1B--4},year={2020},organization={Optica Publishing Group},}
Optica
Obtaining Images by Measuring Time
Alex Turpin , Gabriella Musarra , Valentin Kapitany , Francesco Tonolini , Ashley Lyons , Ilya Starshynov , and 5 more authors
@article{turpin2020obtaining,title={Obtaining Images by Measuring Time},author={Turpin, Alex and Musarra, Gabriella and Kapitany, Valentin and Tonolini, Francesco and Lyons, Ashley and Starshynov, Ilya and Murray-Smith, Roderick and Faccio, Daniele and Villa, Federica and Conca, Enrico and others},journal={Optics and Photonics News},volume={31},number={12},pages={50--50},year={2020},publisher={Optica Publishing Group}}
bioRxiv
Wide-field Fluorescence Lifetime Imaging Microscopy with a High-Speed Mega-pixel SPAD Camera
V Zickus , M-L Wu , K Morimoto , V Kapitany , A Fatima , A Turpin , and 5 more authors
@article{zickus2020wide,title={Wide-field Fluorescence Lifetime Imaging Microscopy with a High-Speed Mega-pixel SPAD Camera},author={Zickus, V and Wu, M-L and Morimoto, K and Kapitany, V and Fatima, A and Turpin, A and Insall, R and Whitelaw, J and Machesky, L and Bruschini, C and others},journal={bioRxiv},pages={2020--06},year={2020},publisher={Cold Spring Harbor Laboratory}}
