Calcium imaging analysis – how far have we come? [version 1; peer review: 3 approved with reservations]

Techniques for calcium imaging were first achieved in the mid-1970s, whilst tools to analyse these markers of cellular activity are still being developed and improved. For image analysis, custom tools were developed within labs and until relatively recently, software packages were not widely available between researchers. We will discuss some of the most popular, alongside our preferred, methods for calcium imaging analysis that are now widely available and describe why these protocols are so effective. We will also describe some of the newest innovations in the field that are likely to benefit researchers, particularly as calcium imaging is often an inherently low signal-tonoise method. Although calcium imaging analysis has seen recent advances, particularly following the rise of machine learning, we will end by highlighting the outstanding requirements and questions that hinder further progress, and pose the question of how far we have come in the past sixty years and what can be expected for future development in the field.


Introduction
The ability to image calcium ion (Ca 2+ ) dynamics in cells has long been of interest, particularly in the neurosciences, where it can be used as a marker for neuronal excitability. The origins of calcium imaging began in the mid-1970s (Blinks et al., 1976;Moisescu et al., 1975), however the most Ca 2+ specific BAPTA-based dye was developed in 1980 by Roger Tsien, and its derivatives are still used today (Tsien, 1980). In the past forty years, the methods available for measuring Ca 2+ fluxes in cells have expanded to include ratiometric, fluorescence lifetime, or fluorescence intensity, based reporters, and genetically-encoded options (Miyawaki et al., 1997;Ohkura et al., 2005) alongside dyes. The use of microscopy modalities has also advanced to include light-sheet microscopy (Huisken et al., 2004) for long-term imaging, and two-photon microscopy (Denk et al., 1990) for deep tissue and cell specific uncaging techniques.
Calcium imaging is an inherently noisy method due to the high spatiotemporal information desired from a sample often showing low signal-to-noise alongside drift or cell movement, particularly for living organisms. In recent years, a number of software packages have been written for individual aspects of the commonly used pipeline in calcium imaging analysis ( Figure 1). This processing pipeline includes image denoising, motion correction, classification for cell identification, and quantification of calcium signals.

Denoising
Although denoising is not a required step in the pipeline, effective denoising can improve the subsequent steps by artificially enhancing signal-to-noise. Traditionally, image denoising has been based on local averaging approaches, such as the application of a Gaussian smoothing filter (Buades et al., 2005;Lindenbaum et al., 1994). Other local filter methods include least mean squares filter (Haykin & Widrow, 2003), anisotropic filters (Perona & Malik, 1990) and in the frequency domain, Wiener filters (Wiener, 1950) and wavelet thresholding methods (Donoho, 1995).
Local methods are computationally light but have clear limitations. Firstly, the averaging often involved in local methods introduces blur, rendering features to be less defined. Secondly, they do not perform well for high noise levels, since the correlations between neighbouring pixels deteriorate (Shao et al., 2014).
Non-local filters solve some of these problems by using selfsimilarity of natural images beyond neighbouring pixels (Shao et al., 2014). The first method to propose this is the non-local means method (Buades et al., 2005), in which patches are restored by weighted averaging of all other patches in an image. Since then, there have been a number of improvements such as invariance to patches that are rotated or mirrored with respect to each other (Grewenig et al., 2011), improved computational efficiency, and automated parameter tuning and extension to 3D image stacks (Coupé et al., 2008). Although non-local filters are better at high noise levels, they will typically lead to artefacts like over-smoothing (Shao et al., 2014). A modern, well-balanced and state-of-the-art non-local method is ND-SAFIR, which is specifically geared towards application in fluorescence microscopy imaging (Boulanger et al., 2010). ND-SAFIR is a powerful method for removing Poisson-Gaussian noise, which is based on non-local means denoising (Buades et al., 2011) to first use a variance stabilisation step, followed by spatial and temporal patch-based weighted averages of intensity values. The method is widely applicable between experimental samples and can be used directly for 2D+t and 3D+t datasets.  supervised learning approach. However, this requires ground truths to be available for model training, which may be difficult to obtain in practice. A different approach was developed in noise2noise (Lehtinen et al., 2018), where instead of learning the mapping from noisy images to clean targets, the model is trained with other noisy images as targets. The images must be corresponding pairs displaying the same objects but with independent noise. Assuming the noise sources underlying the images have zero-mean distributions, the weights of the network will then converge during training to the same values as a network trained with clean targets because the noise that manifests in the weights cancels out. A more recent method, noise2void (Krull et al., 2019), aims to resolve this issue of needing ground truths, by using self-supervised learning. Here, the network is optimised to predict the value of each pixel from the values of neighbouring pixels in an image, thus requiring no separate ground truths.

Motion correction
Motion correction can be split into two main categories, which may be selected depending on the experimental model. Many samples will face drift during imaging or shift when imaging the same field of view over multiple days. which can be well rectified using standard registration methods (Thévenaz et al., 1998).
More complex motion such as organism movement can be harder to correct as it is often non-uniform, over a large area, and causes movement in-and-out of the focal plane. These require non-rigid registration methods or motion tracking. A commonly used example available in Python and MATLAB is Non-Rigid Motion Correction, NoRMcorre (Pnevmatikakis & Giovannucci, 2017), which uses patch-based field of view registration whereby separate images are then merged by smooth interpolation. The popularity of NoRMcorre may in part be due to its general applicability.
Two correction methods have been produced for 2-photon in vivo imaging in awake rodents, one based on the Lucas-Kanade (gradient descent) image registration algorithm using MathWorks® MATLAB platform (Greenberg & Kerr, 2009), the other using a Hidden Markov Model (Dombeck et al., 2007). Although effective, these methods have not been packaged for easy implementation and are reliant in cells remaining in the x-and y-dimensions as it cannot track following movement between z-axes. In cases with z-axis movement, trackingbased methods may be more reliable, and specialist options exist using control theory and machine learning approaches

Classification
Classification can be achieved through pixel-or object-based segmentation. Pixel-based methods map each pixel to a class according to the spectral similarities. Popular pixel-based methods for calcium image analysis include Maximum Likelihood Classification (MLC) or Otsu thresholding to separate 'light' and 'dark' clustered pixels (Otsu, 1979) as used as part of the SIMA Python package ROI pipeline (Kaifosh et al., 2014).
Object-based segmentation is a two-step process using both spectral and spatial/contextual information to group pixels into objects which are then classified. CaImAn is an open-source classification method based on convolutional neural networks (Giovannucci et al., 2019). It was packaged into EZcalcium in an effort to improve usability by providing a GUI in MathWorks® MATLAB (Cantu et al., 2020). However, using limited CaImAn function in EZcalcium does not easily allow for segmentation of more complex structures or large organelles or clusters of cells and is better for somas or smaller, less complex areas. Cellpose is another generalist, deep learningbased segmentation method that uses entirely open source packages in Python with a GUI to aid implementation. There is also a web-based option for testing Cellpose, which makes it very easy to use (Stringer et al., 2020), though it too can be limited at detecting more complex cell shapes such as dendrites and axons.
DenoiSeg is an extension of Noise2Void that offers an endto-end neural network, which is jointly optimised to denoise and segment images. The denoising capability is learnt by the self-supervised learning principle that noise2void introduced (Krull et al., 2019). By combining this with a supervised learning approach using a few annotated ground truths of segmentation maps, the final segmentation performance ends up performing better than without co-learning, i.e. having two separate networks perform the respective tasks (Buchholz et al., 2020).
Cell classification methods have been discussed with the conclusion that 'learning-based methods score among the best-performing methods, but well-optimized traditional methods can even surpass these approaches in a fraction of the time' (Vicar et al., 2019).

Quantification
The aim of each step is for signal extraction to allow a quantitative output from the images of calcium signals. The most commonly used measure is the relative fluorescence variation (ΔF/F0) for classified cells. Packages will therefore either provide this data for further analysis, or provide a direct plot. Background subtraction may need to be considered as not all packages will take this into account.  EZcalcium is one of the most intuitive options, which has improved the usability of CaImAn, NoRMCorre, but again seems best suited to analyse cell bodies.
It therefore seems that perhaps some of the biggest advances could be made by designing packages for detecting neuritic structures or organelles and improving the spatial resolution of the analysis to be intracellular, such as has been used for calcium sparks (Berens et al., 2018). On the other end of the scale, pipelines for functional imaging in organisms such as zebrafish, C. elegans and Drosophila, where motion correction is often required and improved analysis for connectomics purposes are much needed.
As the application of machine learning in calcium imaging analysis matures, a higher level of automation and throughput for analysis tasks can be expected to follow. This will be enabled by more generalised and robust machine learning models. The barrier to training and deploying these methods will also reduce as more research is made into few-shot learning (using small training datasets) in addition to training approaches such as self-supervised and unsupervised learning.

Data availability
No data are associated with this article. Overall the review is a helpful resource to anyone aiming to analyze their calcium imaging data. However a clearer understanding of which resource is better suited to which model system or type of analysis will be a helpful addition (maybe some examples will be particularly insightful). The authors mention Suite2P and EZCalcium as the top analysis tools available however it is not clear which one the reader should pick. Furthermore both Suite2P and EZCalcium come with a user interface which means other than downloading either python or MATLAB no programming knowledge is needed. This fact is not clear in the review.

Open Peer Review
One reference is mentioned as "Balaji, UCLA". The authors should specify if indeed that is a personal communication or perhaps an unpublished tool.

Are the conclusions drawn appropriate in the context of the current research literature? Yes
Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Reviewer Report 24 May 2021 https://doi.org/10.5256/f1000research.54953.r84794 © 2021 Zeiger W. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

William Zeiger
Department of Neurology, University of California, Los Angeles (UCLA), Los Angeles, CA, USA In their review "Calcium imaging analysis -how far have we come?", Robbins et al. present a concise overview of the basic steps involved in transforming raw microscopy images from calcium imaging experiments to quantifiable data. This is an extremely important topic with broad relevance to many fields. As such, the review should draw considerable interest. The review is well written. It is logically organized, with the analysis pipeline broken down into 4 distinct steps, each of which is discussed separately. The language is easy to follow for a broad audience of varying degrees of expertise. Many of the most popular software implementations are covered at least to some degree. Overall, I think this will be a useful and important review with significant impact.
However, I do have some suggestions that I believe could strengthen the manuscript. My main concern is that the target audience for the review is not clear. Calcium imaging is broadly used in many areas of the life sciences, including both in vitro and in vivo preparations, across multiple species, multiple tissues, and widely varying spatial scales (sub-cellular, cellular, bulk tissue signals). Given that much of the review is devoted to denoising and motion correction (problems which are relatively minor in in vitro preparations with no organismal movement and relatively high signal-to-noise), and that quantification focuses on spike detection, the review feels most well suited to in vivo two-photon imaging applications in the brain. If this was the intention, it would benefit the review to make this more explicit and provide more discussion tailored to this technique, particularly in the quantification section. If this assumption is not correct and the review is meant to be targeted at a more general audience, I would suggest the introduction be expanded to include at least a brief discussion of the types of calcium indicators available (ratiometric, FRET, fluo, GCaMP, etc) and imaging techniques in use (widefield fluorescence, confocal, 2P, miniscopes, etc). This is important for a beginner/generalist audience as the indicator and imaging technique used will strongly influence the subsequent processing pipeline needs.
In addition to this primary concern I have a few other suggestions that would improve the scope of the review: The quantification section should be significantly expanded. This step is the shortest in description, yet arguably can be fraught with the most pitfalls to which investigators fall ○ prey.
CaImAn is included under the "classification" section, but includes modules to do more than just classify, including motion correction and registration across imaging sessions ○ Spike detection is mentioned, but it should be made explicit that spike detection is completely dependent on knowledge of ground truth about how specific indicators under specific imaging conditions relate and cannot be readily generalized across labs/indicators/preparations. See this recent preprint (Rupprecht et ○ fail to justify why these steps are needed in the calcium processing and analysis pipeline. An 'introduction' sentence or paragraph for each section is needed.
The acronym BAPTA should be defined before it is used.
In the motion-correction section, more distinction should be made between 'camera-based' motion correction, where the whole image is captured simultaneously, and 'scanning-based' motion correction. There are distinct differences in capabilities and limitations for each approach (for example, scanning-based imaging can suffer from 'warping' which affects camera-based imaging much less, but can also sample regions of interest as opposed to the whole image, speeding up imaging to overcome motion artifacts). The authors successfully reference the original papers for the different approaches to denoising, motion correction, and classification, but they should also include the references for when the techniques were first applied and used in calcium image analysis, as this is the focus of the review.
The standard of presentation is OK, but should really be improved before 'publication'. The first sentence of the abstract is difficult to parse and is not a complete statement. Furthermore, throughout the manuscript there are several difficult-to-parse sentences and some that are ungrammatical (e.g. "to include ratiometric, fluorescence lifetime, or fluorescence intensity, based reporters, and genetically-encoded options […] alongside dyes", "rendering features to be less defined.", "ND-SAFIR is a powerful method for removing Poisson-Gaussian noise, which is based on non-local means denoising […] to first use a variance stabilisation step, followed by spatial and temporal patch-based weighted averages of intensity values", "over multiple days. which can be well rectified using standard registration methods" and so on). Finally, the structure could be improved ("Motion correction can be split into two main categories" really should be followed by an explanation of what those two categories are, and background subtraction is alluded to in the "Quantification" section but not elaborated upon). Other more minor adjustments that are recommended (but not required) include: It would be nice to see a definition of Poisson-Gaussian noise since this is unlikely to be understood by many readers. Similarly clear definition of a 'patch' would be helpful, as would explanation of the difference between local and non-local methods.
As stated previously, further expansion of the methods used for the analysis of calcium signals, such as calculation of dF/F and spike sorting, that are touched on in the quantification section, would be helpful. This section is very brief but would seem to be the section of most interest to the general reader.
The authors state the paper will describe the methods used for calcium imaging analysis however the focus of the paper is more on the pre-processing methods rather than the analysis. It would be nice to elaborate further on what kinds of analyses are performed on the data after preprocessing -how these calcium traces can be used to infer useful neurobiological information.
In summary, this is a brief but well-focussed review on a topic that is of interest to several researchers and thus should be formally published after the necessary revisions have been made.

Is the topic of the review discussed comprehensively in the context of the current literature? Yes
Are all factual statements correct and adequately supported by citations? Yes

Is the review written in accessible language? Partly
Are the conclusions drawn appropriate in the context of the current research literature? Yes Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Microscopy, instrument development, automation and data analysis, neurophotonics, biophotonics We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com