F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.27112.1 Software Tool Article Articles MosaicExplorerJ: Interactive stitching of terabyte-size tiled datasets from lightsheet microscopy

[version 1; peer review: 2 approved with reservations]

 Tosi Sébastien Conceptualization Data Curation Formal Analysis Investigation Methodology Project Administration Software Supervision Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-8348-2778 a 1 Bardia Lídia Resources Validation Writing – Original Draft Preparation Writing – Review & Editing 1 Barallobre Maria Jose Resources Validation Writing – Review & Editing 2 Muñoz-Barrutia Arrate Resources Validation Writing – Review & Editing 3 4 Soto-Montenegro María Luisa Resources Validation Writing – Review & Editing 4 5 Colombelli Julien Data Curation Funding Acquisition Investigation Project Administration Resources Supervision Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-2784-4276 1 Institute for Research in Biomedicine - IRB Barcelona, the Barcelona Institute for Science and Technology - BIST, Barcelona, Spain Department of Developmental Biology, Instituto de Biología Molecular de Barcelona, CSIC, Barcelona, Spain Departamento de Bioingeniería e Ingeniería Aeroespacial, Universidad Carlos III de Madrid, Leganés, Spain Instituto de Investigación Sanitaria Gregorio Marañón, Madrid, Spain CIBER de Salud Mental (CIBERSAM), Madrid, Spain sebastien.tosi@irbbarcelona.org

No competing interests were disclosed.

 10 11 2020 2020 9 1308 2 11 2020 Copyright: © 2020 Tosi S et al. 2020 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

We introduce MosaicExplorerJ, an ImageJ macro to stitch 3D tiles from terabyte-size microscopy datasets. As opposed to existing software, stitching does not require any prior information on the actual positions of the tiles, sample fiducials, or conversion of raw TIFF images, and the stitched images can be explored instantly. MosaicExplorerJ was specifically designed to process lightsheet microscopy datasets from optically cleared samples. It can handle multiple fluorescence channels, dual-side lightsheet illumination and dual-side camera detection.

 3D stitching Lighsheet microscopy Tiled scan Mosaic ImageJ Ministerio de Ciencia, Innovación y Universidades RTC2017-6600-1 PI17/01766 Instituto de Salud Carlos III European Regional Development Fund Ministerio de Economía y Competitividad TEC2016-78052-R The preparation of some of the datasets that were used to test MosaicExplorerJ was partially funded by project TEC2016-78052-R from the Spanish Ministry of Economy and Competitiveness and RTC2017-6600-1 from Ministry of Science, Innovation and Universities, as well as project PI17/01766 from Ministerio de Ciencia, Innovación y Universidades, Instituto de Salud Carlos III (co-financed by European Regional Development Fund (ERDF), “A way of making Europe"). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Introduction

A number of open source tools are available to stitch mosaics from optical microscopy 3D tiled scans 1– 4 but they all implement automated algorithms, potentially converging to a suboptimal solution. This is especially likely if the initial positions are far from the optimal positions, or if the data suffers from unexpected artifacts. Even worse, this situation can be difficult to detect in practice since these tools bring no or scarce support to check the results and correct for errors manually. Additionally, some software limits the size of the datasets 1, 2 , or requires the data to be redundantly converted to ad-hoc formats 3 . Finally, none of these solutions handles both dual-side illumination and dual-side camera detection, two useful lightsheet microscopy 5 modalities that can advantageously be combined ( Figure 1, Left). We developed MosaicExplorerJ to address these shortcomings and bring a complementary alternative to ImageJ BigStitcher 4 , the reference in the field.

Left: Illuminating the sample and collecting the light from both sides enables to image a sample by lightsheet microscopy in the best conditions. Middle: Overlapping tiles (green, blue, red) from a 3D mosaic are shifted axially to compensate for the tilt of the lightsheet (here exaggerated around Y axis). Right: Reconstructed mosaics from both cameras are aligned before stacking their best section.

 Methods Implementation

Whereas stitching the tiles of confocal microscopy datasets chiefly consists in compensating for the scanning head to sample stage tilt, stitching lightsheet microscopy datasets is compounded by the fact that the lightsheet is not necessarily perfectly collinear to the object plane of the detection objective. This can lead to 1) a distortion of the aspect ratio of the images (often negligible), 2) an apparent axial displacement of the tiles while moving across the mosaic. This second effect can be simply compensated by axially offsetting the 3D tiles accordingly ( Figure 1, Middle), but additional lightsheet non-uniformity (or lateral misalignment) can lead to differences in the features visible in the regions of tile overlap; potentially weakening correlation based algorithms. To address these issues, MosaicExplorerJ does not implement any automated stitching but instead assists the user in visually aligning the tiles along their possible degrees of freedom. This is basically achieved by following a step-by-step procedure to compensate for the misalignments highlighted in the regions of overlap between adjacent tiles.

 Operation

ImageJ/Fiji should be installed and MosaicExplorerJ run from ImageJ macro editor. If no other data is available, the software can be tested with the data provided.

First, the mosaics from both cameras (and illumination sides) should be aligned independently before being aligned together. This first operation can be as simple as joining two matching features between two adjacent tiles ( Figure 2B), while compensating lightsheet tilt ( Figures 2C, 2D) and the axial wobbling of the motors forming the mosaics might respectively require to adjust the axial shift of the tiles forming the top left corner or the first row and first column of the mosaic. Dual-camera alignment includes a calibration step to compensate for discrepancy between the magnifications of both detection objectives ( Figure 1, Right). Finally, the user can save the overall mosaic alignment for further inspection, or export the stitched dataset as a TIFF series. Several tile blending modes are available.

A: Two mosaics (2×8 tiles each with a full overlap in the central column along Y) from a dual illumination side scan aligned with MosaicExplorerJ, the arrows show the directions of the light sheets (the view is turned and cropped to fit the figure). Scale bar: 1 mm. B: Zoomed region from the red square prior to adjusting the XY positions of the tiles but after axial shift (Z) correction. The user joins two matching features (white arrow) from two adjacent tiles. C: The XY positions of the tiles are adjusted based on the previous user input and for correct alignment matching features are highlighted in white in the region of overlap. D: Same view but prior to axial shift correction to compensate for lightsheet tilt: no matching features are apparent in the region of overlap. Scale bar: 100 µm.

 Testing

MosaicExplorerJ has been extensively tested by stitching several large lightsheet microscopy datasets acquired from diverse optically cleared biological samples (see Data availability).

 Use cases

All datasets were successfully aligned, each time in under 30 minutes. The results were checked visually by scrolling through the slices and ensuring that the alignment was correct in the regions of overlap between the tiles. The dataset Brain2_izq_2x8Mosaic_LeftSide_300GB was also aligned by BigStitcher, leading to similar results both visually and quantitatively (Extended Data Table S1 6 ). It took about 2h30 to process this dataset with BigStitcher (from 3D TIF tiles to computed alignment), excluding the conversion from TIFF series (the original format of this dataset) to 3D TIFF tiles. This time is expected to scale at least linearly with increasing dataset size for BigStitcher while alignment time in MosaicExplorerJ is relatively constant, and mostly conditioned by the degrees of freedom of the 3D tiles. After alignment, the exportation of the stitched images to TIFF series could be achieved in a comparable time with both tools.

 Conclusion

MosaicExplorerJ brings a complementary alternative to BigStitcher, and presents a number of advantages (Extended Data S2 and Table S2 6 ). No fiducials or detectable feature points are required, which makes the tool robust, versatile, and compatible with large optically cleared samples for which introducing fiducials is virtually impossible. The processing is fast, and terabyte-size datasets can be explored on the fly without conversion to an intermediate format, even on laptop computers with limited memory. Finally, dual-side camera detection is supported and all alignment steps are performed visually, which brings direct control and feedback both on the imperfections of the datasets and on the quality of the results, minimizing the risk of leaving coarse errors unnoticed.

 Data availability Underlying data

A complete description of the datasets used to test MosaicExplorerJ, including sample preparation and imaging can be found in Extended Data S1. The full datasets are too large (1.5TB) to feasibly host on a data repository; however, datasets can all be accessed publicly on the IRB Barcelona Google Drive at: https://bit.ly/37iocrP.

 Extended data

Zenodo: MosaicExplorerJ F1000Research article extended data. https://doi.org/10.5281/zenodo.4156030 6 .

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

 Software availability

Source code and documentation are available from: https://github.com/SebastienTs/MosaicExplorerJ

Archived source code at time of publication: https://doi.org/10.5281/zenodo.4106848 7 .

License: GNU General Public License v3.0.

 Acknowledgements

This publication was supported by COST Action NEUBIAS (CA15124), funded by COST (European Cooperation in Science and Technology). MJB acknowledges the support of Jérôme Lejeune Foundation.

 Emmenlauer M Ronneberger O Ponti A : XuvTools: free, fast and reliable stitching of large 3D datasets. J Microsc. 2009;233(1):42–60. 19196411 10.1111/j.1365-2818.2008.03094.x https://imagej.net/Grid/Collection_Stitching_Plugin . Bria A Iannello G : TeraStitcher - a tool for fast automatic 3D-stitching of teravoxel-sized microscopy images. BMC Bioinformatics. 2012;13:316. 23181553 10.1186/1471-2105-13-316 3582611 Hörl D Rojas Rusak F Preusser F : BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples. Nat Methods. 2019;16(9):870–874. 31384047 10.1038/s41592-019-0501-0 Huisken J Swoger J Del Bene F : Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science. 2004;305(5686):1007–1009. 15310904 10.1126/science.1100035 Tosi S Bardia L Colombelli J : MosaicExplorerJ F1000Research article extended data (Version 1.0). Zenodo. 2020. http://www.doi.org/10.5281/zenodo.4156030 Tosi S : SebastienTs/MosaicExplorerJ: First_Release (Version 1.0). Zenodo. 2020. http://www.doi.org/10.5281/zenodo.4106848 10.5256/f1000research.29948.r75056 Reviewer response for version 1 Pampaloni Francesco 1 Referee https://orcid.org/0000-0003-4357-7320 Buchmann Institute for Molecular Life Sciences, Goethe University, Frankfurt am Main, Germany

Competing interests: No competing interests were disclosed.

 11 12 2020 Copyright: © 2020 Pampaloni F 2020 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Overview

The MosaicExplorerJ ImageJ macro-code described in the manuscript is a useful tool to stitch very large data sets recorded with light sheet microscopy. Indeed, the amount of data acquired in a typical session of light-sheet imaging of optically cleared organs (e.g. the mouse brain), reaches easily the TB range with up-to-date equipment. Therefore, user-friendly and straightforward data navigation and stitching software is needed by the scientific community. MosaicExplorerJ, at variance with automatic stitching algorithms such as BigStitcher, it is a fully manual tool, requiring the user manually perform all the alignment steps in order to compensate the tiles mismatch, e.g. the tiles axial shift arising from a tilted light sheet. Moreover, also at variance with BigStitcher, multi-tiff image stacks can be opened directly without conversion.

I recommend the manuscript for idexing with minor changes and additions as described in the following remarks.

Remarks

The dataset deposited by the authors on the IRB Barcelona server allows readily testing the macro. The macro runs from the macro editor. The dataset is quickly loaded into the main window. In order to properly upload the tiles, the different files have to be named sequentially following the explained rule. After opening the dataset, the alignment and stitching are performed by manually setting the shifting parameters (xy translation and axial alignment along z) shown in the control panel.

In general, a first-time user can achieve the stitching of the mosaic 3D dataset as described in the manuscript by following the steps outlined in the documentation available on GitHub. However, the learning curve is quite steep and some effort and time is necessary by the user in order to understand the working of the macro and to achieve the result. In order to allow for a smoother learning curve, I strongly recommend the authors to upload videos illustrating the stitching procedure for two datasets at increasing difficulty.

Particularly, the following step described in the documentation (p. 3, end) is not clearly explained, and should be better illustrated with a further figure and in a video:

“For linear correction, it is recommended to use the top left tile as reference and gradually tweak ZxCor from the tile to the right, until some matching features are apparent in the overlap region. Then ZyCor can be adjusted, this time by considering the tile below. The adjustment of the Z offsets can be performed by typing values in the Panel or by following the procedure described below for free correction.”

I also recommend to include the documentation in the manuscript extended material as well, and not exclusively in GitHub, as the macro “instructions” are an essential part of the work.

Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Yes

Is the rationale for developing the new software tool clearly explained?

Yes

Is the description of the software tool technically sound?

Yes

Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Partly

Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes

Reviewer Expertise:

Light Sheet Microscopy, 3D cell biology, stem cell and tumor organoids.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Tosi Sebastien

Competing interests: No competing interests were disclosed.

 21 1 2021

Thank you for reviewing the article and for pointing out these weaknesses of the documentation. We have tried to substantially improve the documentation accordingly and we have created five video tutorials to help illustrate all the steps described in the documentation. Please see our detailed answers below.

> In general, a first-time user can achieve the stitching of the mosaic 3D dataset as described in the manuscript by following the steps outlined in the documentation available on GitHub. However, the learning curve is quite steep and some effort and time is necessary by the user in order to understand the working of the macro and to achieve the result. In order to allow for a smoother learning curve, I strongly recommend the authors to upload videos illustrating the stitching procedure for two datasets at increasing difficulty.

We have included five video tutorials that illustrate the main operations and complement the documentation. These videos are available from the GitHub repository and from the new Extended Data  S3.

> Particularly, the following step described in the documentation (p. 3, end) is not clearly explained, and should be better illustrated with a further figure and in a video: “For linear correction, it is recommended to use the top left tile as reference and gradually tweak ZxCor from the tile to the right, until some matching features are apparent in the overlap region. Then ZyCor can be adjusted, this time by considering the tile below. The adjustment of the Z offsets can be performed by typing values in the Panel or by following the procedure described below for free correction.”

We have completely rewritten this part of the documentation and the new video tutorial S3-V2 is fully dedicated to explaining this critical step.

> I also recommend to include the documentation in the manuscript extended material as well, and not exclusively in GitHub, as the macro “instructions” are an essential part of the work.

We created a new Extended Data  S3 with links to the documentation of the software and to the five new video tutorials. Both are also available from the GitHub repository.

 10.5256/f1000research.29948.r74555 Reviewer response for version 1 Preibisch Stephan 1 2 Referee https://orcid.org/0000-0002-0276-494X Berlin Institute for Medical Systems Biology, Berlin, Germany HHMI Janelia, Ashburn, VA, USA

Competing interests: No competing interests were disclosed.

 7 12 2020 Copyright: © 2020 Preibisch S 2020 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The authors present MosaicExplorerJ, an ImageJ macro for manual alignment of tiled lightsheet acquisitions.

I think it can be useful as it provides simple, editable source code. I personally found it quite hard to use, but more practice might make it more efficient. But more explanations of the parameters and keyboard shortcuts in the application would certainly be helpful (press F1 or similar).

Most importantly, the paper should clearly state the limitations of the software and correct wrong statements about the comparison to other tools, in particular BigStitcher from my point of view.

Not mentioned limitations (or implicitly assumed ones):

all files need to have the same dimensions (also in z)

only 3D translation models are supported

no chromatic aberration correction

maximal xy output size is 2^31 pixels, the limitation of ImageJ (around 47000x47000 pixels)

 Incorrect comparisons that should be corrected:

1) “As opposed to existing software, stitching does not require any prior information on the actual positions of the tiles, sample fiducials, or conversion of raw TIFF images, and the stitched images can be explored instantly.”

That is the same in BigStitcher, conversion to HDF5 is not necessary, TIFFs can be virtually loaded and displayed instantaneously 

2) “but they all implement automated algorithms … Even worse, this situation can be difficult to detect in practice since these tools bring no or scarce support to check the results and correct for errors manually.”

BigStitcher provides user friendly methods for manual & interactive alignment (3D translation): https://imagej.net/BigStitcher_manual_translation

 3) “Finally, none of these solutions handles both dual-side illumination and dual-side camera detection, two useful lightsheet microscopy”

BigStitcher supports dual-side illumination and dual-side camera detection (e.g. Supplementary Figure 17)

 4) S2-Comparison with BigStitcher:

3D TIFF tiles can be natively used through virtual loading, live display in BDV even. HDF5 conversion is optional (yet very useful)

2D TIFF slices are also supported, but it's a quite hidden feature]

BigStitcher supports Dual-camera support

Physical model constrained grid alignment is also possible, see link above, even interactive & manual

Automatic color highlighting of tiles mismatch can be done in BigStitcher

BigStitcher additionally supports per-tile intensity correction using Affine 1D models

Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Partly

Is the rationale for developing the new software tool clearly explained?

Partly

Is the description of the software tool technically sound?

Partly

Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes

Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes

Reviewer Expertise:

Image analysis, image registration, lightsheet

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Tosi Sebastien

Competing interests: No competing interests were disclosed.

 21 1 2021

First of all, we thank the reviewer for his thorough comments and we would like to apologize for overlooking some of BigStitcher's features that we were genuinely unaware of. Also, as rightly pointed out by the reviewer, some limitations of MosaicExplorerJ were not discussed in the article. We have remedied this, as described point by point in our answer, and we hope that the features and the operating workflow of the two applications are now reflected more accurately in the fully updated Extended Data  S2.

> More explanations of the parameters and keyboard shortcuts in the application would certainly be helpful (press F1 or similar).

We have tried our best to improve this aspect in MosaicExplorerJ (v1.5, now on GitHub): a help window can now be displayed by pressing F1, contextual information is provided in ImageJ status bar (and strategically in some of the dialog boxes), and the manual operation to perform axial adjustment of the tiles has been simplified. Finally, five video tutorials ( bit.ly/2KY0BDN) were created to illustrate the main operations and to complement the documentation.

> Not mentioned limitations (or implicitly assumed ones) of MosaicExplorerJ:

1. All files need to have the same dimensions (also in z)Actually, this is not strictly required as missing tiles or tiles missing Z slices (for 2D TIFF series mode) will not trigger an error but the corresponding images will just be displayed black. However, to answer this comment, we now state in the Abstract of the article (and in Extended Data  S2) that the tiles should be organized on a regular 2D grid.

2. Only 3D translation models are supported

Yes, only these operations are supported for a single camera dataset but 2D similarity (rotation, scaling, translation) is performed (after 3D translations) when registering the mosaics from two opposing cameras (as described in Figure 1).

3. No chromatic aberration correction

We now explicitly state in Extended Data  Table S2 that chromatic aberrations correction is not supported by MosaicExplorerJ (while it is supported by BigStitcher).

4. Maximal xy output size is 2^31 pixels, the limitation of ImageJ (around 47000x47000 pixels)

We now explicitly state in Extended Data  Table S2 that mosaics stitched by MosaicExplorerJ are limited to about 47.000 x 47.000 pixels (and eight channels) while BigStitcher sets no limit. We haven’t included this information in the text of the main article as we believe that it is not really limiting in practice.

Incorrect comparisons that should be corrected:

1) “As opposed to existing software, stitching does not require any prior information on the actual positions of the tiles, sample fiducials, or conversion of raw TIFF images, and the stitched images can be explored instantly.”

That is the same in BigStitcher, conversion to HDF5 is not necessary, TIFFs can be virtually loaded and displayed instantaneously.

We apologize for this misleading statement and we were unaware that tiles could be loaded as a set of 2D TIFF series in BigStitcher. Still, we believe that HDF5 conversion is implicitly required for efficient operations when handling TB datasets, both for interactive 3D viewing and to enable a practical processing time (especially when several rounds of tile placement / parameters optimizations have to be performed to get to satisfying results). We updated this paragraph of the text to reflect this idea, and we stated in Extended Data  Table S2 that while not strictly required HDF5 conversion is desirable to speed up operations.

2) “but they all implement automated algorithms … Even worse, this situation can be difficult to detect in practice since these tools bring no or scarce support to check the results and correct for errors manually.”

BigStitcher provides user friendly methods for manual & interactive alignment (3D translation):  https://imagej.net/BigStitcher_manual_translation

We were aware of this feature but, as far as we can tell, tiles are first automatically laid down to a regular grid (with fixed overlap), and then only individual tiles (or whole groups) can be interactively moved. This feature is certainly useful to coarsely set the initial positions of the tiles but not so much to finely adjust the mosaic (that would ultimately require to finely move tiles on a tile by tile basis). In turn, thanks to the predefined physical correction models, as few as two pairs of tiles need to be manually adjusted in MosaicExplorerJ to stitch a whole mosaic (with no further processing). We tried to reflect this more accurately in this paragraph and in Extended Data  Table S2.

3) “Finally, none of these solutions handles both dual-side illumination and dual-side camera detection, two useful lightsheet microscopy”

BigStitcher supports dual-side illumination and dual-side camera detection (e.g. Supplementary Figure 17)

We apologize for this plainly wrong statement (we did not know that BigStitcher supported dual-sided cameras as the option is not listed in the image loader). We modified the text to exclude BigStitcher from the set of software not supporting the combination of these two modalities and we reflected these features in details in Extended Data  S2 and Extended Data  Table S2.

4) S2-Comparison with BigStitcher:

3D TIFF tiles can be natively used through virtual loading, live display in BDV even. HDF5 conversion is optional (yet very useful)

2D TIFF slices are also supported, but it's a quite hidden feature

Extended Data  Table S2 has been updated to reflect this.

BigStitcher supports Dual-camera support

Extended Data  Table S2 has been updated to reflect this.

Physical model constrained grid alignment is also possible, see link above, even interactive & manual

For the mosaic stitching part only (excluding cameras registration), MosaicExplorerJ physical correction models include camera tilt (XY translations), lighshsheet tilt (linearly increasing Z translations) and motor axis axial wobble (XY separable Z translations). BigStitcher global tile layout only supports a regular grid with fixed overlap (not accounting for previously mentioned effects). In our view, this is useful to coarsely set the starting positions of the tiles prior to optimization but it is really practical to finely adjust the positions of the tiles of large mosaics with no further processing. We tried to reflect this more accurately in the text and in Extended Data  S2.

Automatic color highlighting of tiles mismatch can be done in BigStitcher

Extended Data  Table S2 has been updated to reflect this. MosaicExplorerJ supports alternate red/green (and cyan/yellow) while BigStitcher supports random colors.

BigStitcher additionally supports per-tile intensity correction using Affine 1D models

Extended Data  Table S2 has been updated to reflect this.