Muscarinic receptor agonists in animal models of psychosis: protocol for a systematic review and meta-analysis

Objectives

We therefore plan a systematic review and meta-analysis on the effects of muscarinic receptor agonists in animal models of relevance for psychosis concerning behavioural and motor outcomes as compared to control conditions and existing D2R-blocking antipsychotics. We will also carefully assess the potential biases of these studies and evaluate the confidence in the evidence, ultimately providing evidence-based information to facilitate future drug development in schizophrenia.

Study eligibility criteria and outcomes

Study design

We will include in vivo animal experimental studies examining any muscarinic receptor agonist versus inactive or active comparison groups in animal models of relevance for psychosis, as detailed below. There will be no restrictions on the inclusion criteria in terms of the randomization, blinding or other factors related to risk of bias, unit of allocation, duration of the study, publication status, year, country and language. We will exclude uncontrolled preclinical experiments, observational studies, and literature reviews.

Animal population and model induction

We will include animals that have undergone laboratory methods to induce psychosis-like behaviours and features. There are numerous models fitting this description, each with varying degrees of validity and unique strengths and weaknesses, but none are considered the gold standard.^23–30 Therefore, we will include the “classical” pharmacological models of psychosis and their behavioural readouts (see “Outcomes”), which have been widely utilized in drug discovery and possess some predictive validity, especially for positive symptoms, including the administration of psychostimulants (e.g., amphetamine, cocaine) or N-methyl-D-aspartate (NMDA) receptor antagonists (e.g., phencyclidine, ketamine, MK-801).^29,31,32 Other methods of induction will also be eligible and can include other pharmacological models, neurodevelopmental methods, lesion methods, genetic models, and combinations of different methods.^24–30 Such relatively broad inclusion criteria were utilized in our previous systematic review,^21,22 and any decision on the eligibility of the psychosis models will be made in consultation with experts in preclinical research (MRM, AB, FJ, FT, IM, SN, AdB, SH, NID, UT).

There will be no restriction on species, strain, age, and sex. Regarding genetic composition, we will include both wildtype animals and those that have undergone genetic interventions, if these interventions belong to eligible methods of induction (see above). Moreover, in the eligible studies, we will extract data from “naïve” animal cohorts (i.e., animals that have not undergone models of psychosis), and animal cohorts that have undergone both models of psychosis and muscarinic receptor antagonism via genetic or pharmacological manipulation. The eligible animal cohorts can be found in Table 1.

We will exclude animals that have undergone induction methods for other specific conditions (e.g., transgenic models of Alzheimer’s disease,³³ the valproic acid-induced model of relevance for autism³⁴ and methods aiming to specifically model depressive-like behaviours, such as animal models of physical, social, or chronic mild stress).³⁵ We will also exclude in vitro, ex vivo, in silico studies and studies in humans. We will however consider extracting data from ex vivo measurements (e.g., Fos expression, autoradiography) following eligible in vivo experiments, potentially analysed in secondary publications (see “Outcomes”).

Interventions

We will include any pharmacological agent acting as an agonist or positive allosteric modulator at any of the five subtypes of muscarinic acetylcholine M1-M5 receptors. There will be no restrictions on their receptor selectivity, pharmacological potency and efficacy (e.g., full or partial agonists), dose, timing of administration relative to the induction method, pharmacokinetic properties, or route of administration, provided the method is suitable for achieving effects in the central nervous system. These pharmacological agents can be administered individually or in combination with D2R-blocking antipsychotics or other medications, and combinations can be considered as distinct interventions. We will also consider data on the effects of muscarinic receptor agonists in the context of muscarinic receptor antagonism (see “Animal Population and model induction”) to evaluate whether the effects depend on the activation of muscarinic receptors or other mechanisms (Table 1).

We will exclude clozapine, an existing antipsychotic acting on multiple neurotransmitter receptor systems and also as muscarinic receptor partial agonist,^2,8 which will be considered as a control intervention (see “Comparison groups”). Clozapine’s metabolite N-desmethylclozapine however, which has low affinity for dopamine receptors and acts as a muscarinic M1/M4 receptor agonist,³⁶ will be considered among the experimental interventions. Additionally, we will exclude pharmacological agents with different mechanisms of action (e.g., nicotinic acetylcholine receptor agonists, acetylcholinesterase inhibitors) and non-pharmacological interventions, including genetic interventions for muscarinic receptor overexpression.

Comparison groups

We will include the following comparison groups: i) inactive control conditions, consisting of animals undergoing models of a method to induce psychosis-like behaviour and features and receiving vehicle (e.g., injection of saline) or no treatment, and ii) active control conditions, consisting of the aforementioned animal cohorts treated with D2R-blocking antipsychotics. D2R-blocking antipsychotics will be defined as those medications listed in the Anatomical Therapeutic Chemical (ATC) classification with a code of N05A, except for lithium (N05AN01, a mood stabilizer primarily used for the treatment of bipolar disorder), pimavanserin (N05AX17, a 5-HT2A antagonist indicated for Parkinson’s disease psychosis but not schizophrenia), and any muscarinic receptor agonists approved by any regulatory body for schizophrenia (which will be considered as experimental interventions in this review), with no further restrictions.

We will also consider data in the eligible studies from sham procedures, consisting of “naïve” animal cohorts not subjected to methods of inducing psychosis-like behaviours or features and receiving either vehicle or no treatment, to calculate normalized mean differences in a sensitivity analysis (see “Effect sizes”).

Outcome measures

There is no gold standard measure of preclinical antipsychotic efficacy due to the limited homology with the clinical symptoms of psychosis and the lack of an established biomarker.^{6,25,26,29–32,37–39} Therefore, we aim to provide a comprehensive assessment of the effects of muscarinic receptor agonists by using data from a broad range of behavioral measures in various animal models of relevance for psychosis (see “Animal population and model induction”). This approach will facilitate the identification of strong clinical canditates.^{24,25,30–32}

We will examine as co-primary outcomes the effects on i) locomotor activity and ii) prepulse inhibition of the acoustic startle reflex, as these have been widely used with some predictive validity to identify antipsychotic effects in animal models of relevance for psychosis (see “Animal population and model induction”).^21,26,31 Furthermore, we will examine additional behavioral measures as secondary outcomes, including potential proxies for positive symptoms (e.g., stereotypies, hallucinatory-like percepts),⁴⁰ negative symptoms (e.g., lack of social interaction, operant-based motivational tasks),^31,41 depressive- (e.g., forced swim, tail suspension, sucrose preference tasks)^31,41 and anxiety-like behaviors (e.g., elevated plus maze), and cognitive function (e.g., tests recommended by the CNTRICS initiative).^30,37,39 The eligibility and any potential grouping of the behavioral measures will be evaluated in collaboration with experts in preclinical research prior to commencing data analysis, and any decisions will be documented.

We will exclude non-behavioural outcomes, such as histopathological and neurobiological measures due to the lack of an established biomarker,⁶ and adverse events due to the inconsistent and scarce reporting in animal studies, as identified in our previous review.²¹ Nonetheless, we will aim to extract their data, if available, and potentially analyse them in secondary publications.

Study identification

We will conduct searches in PubMed, MEDLINE via Ovid, Web of Science, EMBASE, PsychINFO, from database inception using keywords for psychosis, muscarinic agents, and animal filters,⁴² similar to a previous review.^21,22 For muscarinic agents, we will use broad terms like “muscarinic agonists” and the names/synonyms of specific relevant agents identified from IUPHAR/BPS⁴³ and previous reviews.^7,44 There will be no restrictions such as on language or publication date. The search strategies will be developed in collaboration with information specialist (see “Acknowledgment”). A draft search strategy in MEDLINE via Ovid can be found in the extended data, and the final search strategies will be reported according to the PRISMA statement for reporting literature searches (PRISMA-S).⁴⁵

We plan additional searches to improve the coverage of the study identification and identify potentially unpublished studies:

Study selection

At least two independent reviewers will screen in the Systematic Review Facility (SyRF)²³ the de-duplicated records identified in the searches in two phases: i) title and abstract, and ii) full-text screening. Any discrepancy between the two reviewers will be resolved through discussion with a more senior reviewer. If not resolved by discussion, the full-text of the study will be acquired (if at the title/abstract level) or additional information from the original study authors will be obtained (if at the full-text level).

At the title/abstract screening, records will be excluded according to the following hierarchy: i) review articles, ii) not referring to in vivo animal study, iii) not referring to muscarinic receptors or agents acting on them.

At the full-text screening, records will be excluded from the review and/or meta-analysis according to the following hierarchy against the eligibility criteria: i) ineligible study design, ii) ineligible animals/population, iii) ineligible intervention, iv) ineligible comparison groups, v) ineligible outcome measure, vi) inadequate reporting of outcome data. If possible, we will consider the studies excluded in terms of their reported outcomes in the assessment of reporting bias (see “Data synthesis”). The study selection process and the reasons for exclusions at the full-text level will be reported in a flow diagram.⁴⁸

Data extraction

At least two independent reviewers will conduct the data extraction in SyRF²³ using pre-specified data extraction forms that will be adapted from a previous systematic review.²¹ Discrepancies between the two reviewers will be reconciled by a more senior reviewer, or if not possible, by contacting study authors for additional information.

We will extract data regarding study identification (e.g., author names, title, publication year), study design (e.g., risk of bias, reporting completeness), animal population and model induction (e.g., age, sex, species strain, body weight, characteristics of the induction method), experimental and control interventions (e.g., type, dose and timing of the administration of drugs), and outcome measures (e.g., exact name of the behavioural task, methods of measurement, quantitative data). We will seek information from various sources in the following order of priority: i) text and tables, ii) figures using WebPlotDigitizer version 4 (a free and opensource tool distributed under GNU Affero General Public License Version 3),⁴⁹ iii) contacting authors for missing information, and iv) employing imputation methods.

We will extract quantitative data for the outcomes anticipated to be reported as continuous measures. Endpoint and change scores will both be eligible and jointly synthesized.⁵⁰ When both are reported, we will prefer endpoint scores because they do not require baseline assessments and are expected to be the most frequently reported in the eligible animal experiments. We will extract the unit of measurement, mean, standard deviation and the number of participants that these correspond to. We will apply a minus transformation whenever appropriate to harmonize the direction of effects across the extracted data (e.g., a higher score indicating a better outcome). Missing standard deviations will be derived by the following methods according to their order of priority: i) calculation from standard error, ii) estimation from test statistics (e.g., p-values, t-tests, median and ranges), iii) contacting the authors of the original studies for additional information, iv) imputed from the standard deviations of other studies (although this method has not been validated in animal studies that often have small sample sizes).^51,52 If the exact number of animals is not reported,⁵³ we will estimate it with available information (e.g., using the lower boundary of a range, if reported) or consider imputation methods. If dichotomous measures are reported, we will extract the number of animals with the event and the total number of animals analysed. We will exclude studies with imputed data in a sensitivity analysis (see “Sensitivity analysis”).

For crossover trials, we will prefer data from the first phase to avoid carryover effects, but we will also use data from the entire phase by applying appropriate corrections considering the within-subject correlation.⁵⁴ These studies will be excluded in a sensitivity analysis (see “Sensitivity analysis”).

We will use data from any reported time point, but preference will be given for the longest time point following multiple administrations of the intervention over an extended period. If outcomes are measured multiple times after a single administration, we will consider calculating the area under the curve.^22,55

If multiple variations of the same outcome are reported, we will extract and jointly analyse them (see “Data synthesis”).

Assessment of risk of bias and reporting completeness

Two independent reviewers will assess risk of bias using the SYRCLE’s tool considering domains for selection, performance, detection, attrition, reporting and other biases.⁵⁶ We will assign an overall high risk of bias to a study if at least one domain in the SYRCLE’s tool is assessed as having a high risk of bias. Since high-quality reporting is essential for assessing the risk of bias, two independent reviewers will also evaluate the completeness of reporting using a modified version of the ARRIVE Essential 10 checklist.^22,57 This is necessary because the reporting completeness of animal studies is often poor, frequently leading to unclear assessments of risk of bias. Any discrepancies between the two reviewers will be resolved through discussion with a more senior reviewer or by contacting the study authors for additional information.

Assessment of indirectness of animal experiments

We expect that the findings from various animal models of relevance for psychosis will have different degrees of applicability to clinical trial settings. However, there is no established method for assessing their indirectness in the context of a systematic review. Moreover, there is limited synthesized evidence on the validity of animal models of relevance for psychosis, and we expect substantial variability in the methods of modelling and measuring psychosis-like behaviours in animals.^25,26 This makes it challenging to set predefined criteria for assessing the indirectness of animal experiments in the context of psychosis.

Nevertheless, we will aim to use the extracted data to provide an experiment-level judgment of indirectness as “low risk”, “high risk”, or “some concerns”, considering how closely the experiment reflects the clinical trial setting in terms of animal population, model induction, intervention, and outcome. This assessment will inform the evaluation of indirectness domain in the confidence in the evidence (see “Confidence in the evidence”).

To achieve this, we will evaluate the validity in animal experiments and the applicability of the intervention (e.g., treatment over an extended period, initiation of treatment after model induction) based on previous frameworks and checklists.^21,58–62 Specifically, we will apply the framework of Belzung and Lemoine⁶² to assess the following domains (and sub-domains) of validity in animal experiments, i.e., homological (species strain), pathogenic (ontopathogenic, triggering), mechanistic, face (ethological, biomarker) and predictive (induction, remission). This framework can offer a more refined and systematic approach compared to the traditional domains of construct, face and predictive validity, which have often been inconsistently applied in the literature.^25,62,63 However, the exact methods will be determined a posteriori in consultation with experts in preclinical research.

Data synthesis

Planned comparisons

Our main aim is to synthesize data for each outcome and for the comparisons described in Table-1. Meta-analysis will be conducted when there are at least two independent effect sizes for the same outcome, as in our previous systematic review.²¹

We will examine the data and if there is reasonable consistency across the comparisons, we will consider network meta-analysis to examining the comparative effects of the different muscarinic receptor agonists, various D2R-blocking antipsychotics, and inactive control conditions.⁶⁴

Effect sizes

The main effect size will be the standardized mean difference (SMD) due to the varying measures and units of the behavioural outcome measures across the studies. We will also use normalized mean difference (NMD) in a sensitivity analysis.^53,65 If outcomes are reported as dichotomous, we will calculate odds ratios (ORs) and convert them into SMDs using the Hasselblad and Hedges method⁶⁶ to enable their combination with results of continuous measures.

In addition to the estimation of the average treatment effects, we also aim to conduct a meta-analysis of variation to estimate the inter-individual variability of the effects using the variability ratio (VR) or the coefficient of the variation ratio (CVR) in case a mean-variance relationship is expected.^65,67 This analysis will be conducted for the comparison of muscarinic receptor agonists to inactive control conditions.

Data synthesis approach

We will opt for synthesizing the data using multilevel meta-analytic models, which enable handling non-independent data.⁶⁵ We will use a predefined multilevel random-effects structure with nested levels, from higher to lower, for animal strain, study, and experiment, provided there are at least five distinct categories for at least one of the levels, as in our previous systematic review.²¹ For non-independent sampling errors, we will estimate the within-study variance-covariance (VCV) matrix using any reported correlation in the original studies or assuming a correlation of 0.5 (see other assumed correlations in “Sensitivity analysis”).⁶⁵ The restricted maximum likelihood (REML) method will be used to estimate the between-study variance (τ²) and between-study VCV.^65,68 We will adjust the confidence intervals using t- or F-distributions with degrees of freedom appropriate for the multilevel model.⁶⁹

To our knowledge, network meta-analysis has not been widely applied to the synthesis of animal experiments, and we anticipate several challenging issues, including the limited evidence for or against inconsistency, small sample sizes, and non-independent effect sizes.⁶⁰ We will examine whether the assumptions of a network meta-analysis can be fulfilled by comparing the distribution of potential effect modifiers across treatment comparisons and measuring incoherence using statistical tests.⁷⁰ Justifying these assumptions with preclinical data however, might be challenging. If a network meta-analysis is deemed feasible, we will extend the multilevel models and consider covariate-adjusted analysis (see “Exploration of heterogeneity”). The exact methodology will be defined a posteriori, and, if a network meta-analysis is justified, will be thoroughly reported in an amendment of the protocol before conducting it.

We will present the effect sizes with their 95% confidence intervals and prediction intervals.

Exploration of heterogeneity

We will quantify heterogeneity using the variance of the random effects with its components, and the 95% prediction intervals. We will explore potential sources of heterogeneity for each outcome through meta-regression or subgroup analyses, if sufficient data are available, considering the following tentative list of potential effect-modifiers: age, sex, species/strain, comorbidities, characteristics of the model of induction of psychosis-like behaviours (e.g., pharmacological or genetic, severity), muscarinic receptor agonists and their pharmacological characteristics (e.g., mode of action, selectivity for muscarinic or other receptors, potency, efficacy), dose, route of administration, duration of treatment, timing of the intervention (e.g., before or after model induction), co-treatments, characteristics of the study (e.g., publication year, reporting completeness, and risk of bias), and outcome measurements (e.g., different measures for locomotor activity). Additional variables will be considered if they are deemed relevant and there are sufficient data.

We will also aim to standardize doses across drugs to examine dose-effects relationships using the standardized dose as a covariate in a meta-regression. However, any potential standardization method cannot be predefined due to the pharmacological differences among various drugs²¹ and differences across species/strains.⁷¹

Sensitivity analysis

The robustness of the findings for each outcome will be examined through sensitivity analysis by: i) excluding studies with an overall high risk of bias (see “Assessment of risk of bias and reporting completeness”), ii) using normalized mean difference (see “Effect Sizes”), iii) excluding studies with imputed data (e.g., standard deviations, number of animals) and crossover studies reporting data from the entire period, iv) excluding interventions examined in single studies due to potentially inflated effect estimates,⁶⁰ v) assuming correlations of 0.2 and 0.8 in estimating the within-study VCV,⁶⁵ and vi) using a robust variance estimation to obtain cluster-robust standard errors.^65,72

Reporting bias and small-study effects

Publication and other non-reporting biases are highly prevalent in preclinical research, potentially having a substantial impact on the estimated efficacy.^73,74 We will aim to evaluate potential within- and across-study non-reporting biases by adapting the framework for assessing the risk of bias due to missing evidence (ROB-ME) in clinical trials, assigning ratings of “low risk”, “high risk”, or “some concerns”.^75–77 We will include both published and unpublished studies, although the latter may be difficult to find due to the limited adoption of pre-registration protocols. We will evaluate potential reasons of non-reporting of outcome data in studies excluded due to ineligible outcome measures or inadequate reporting of outcome data, although this may be more challenging due to the poor reporting quality of animal studies (see “Assessment of risk of bias and reporting completeness”). Moreover, we will explore small-study effects for each outcome using contour-enhanced funnel plots and multilevel regression-based tests,⁶⁵ using the square root of the sample size, when there are sufficient data from at least 10 studies.

Confidence in the evidence

We will evaluate the confidence in the evidence for each of outcome using a modified version of the GRADE framework^58,59 taking into account the domains of risk of bias (and reporting completeness), indirectness, heterogeneity, imprecision, and reporting bias, similar to our previous systematic review.²¹ Ultimately, we will aim to draw an overall conclusion on the preclinical efficacy of muscarinic receptor agonists by considering the evidence from the different behavioural domains.

Statistical software

Data analysis will be conducted in R statistical software⁷⁸ using the package metafor,⁶⁹ along with other appropriate packages for data cleaning, specific meta-analytic models, and visualization. We will report the complete list of packages used along with their versions in the publication of the results.

Dissemination of information

We plan to publish the systematic review and meta-analysis as open access in peer-reviewed journals, potentially resulting in multiple publications, and present the findings at conferences. Lay language summaries will be prepared and disseminated with the help of patient and relative groups, such as BASTA (Bündnis für psychisch erkrankte Menschen) and ApK (Aktionsgemeinschaft der Angehörigen psychisch Kranker e.V.). We will also make the methods, data, and code publicly available in a GitHub repository.

Study status

As of the date of submission of this protocol on 18.08.2024, we have completed the preliminary searches and piloting of study selection process and started, but not completed, the full searches and the full screening of search results against the eligibility criteria. We have not yet started the data extraction, risk of bias and quality assessment, or data synthesis.

There were no changes from the original PROSPERO registration of the protocol, except for expanding the methods with additional details and deciding not to search Scopus and CINAHL after consulting with the information specialist (which is not expected to affect the coverage of our search). Any additional deviations or modifications will be reported along with the findings and in updates to the PROSPERO registration.