see also Medical statistical analysis

• A discipline which provides methods of finding, appraising & combining data from a range of studies;
• Although bias can intrude in a traditional narrative review, it is more important in meta-analysis because a wealth of apparently disparate data can be reduced to a few simple statistics which are likely to influence treatment or public policy, thus need to assess quality by asking:

1. Did authors work to written protocol?
   • guidelines in advance decrease bias due to decisions made after study group developed a “feel” for the data;
2. Have authors defined research questions clearly?
   • OK to address issues not the primary objectives of individual studies;
   • must be formulated prior to doing the analysis & should be biologically plausible & of clinical importance;
3. Have authors described there search strategy & how studies were chosen for inclusion/exclusion?
   • Medline single search → only 2/3rds relevant citations;
     • ⇒ cross-check bibliographies of those papers;
   • OK to excl. non-English but may add bias thus should at least read abstracts to see if “+ve” or “-ve”;
4. How have authors assessed quality of individual studies?
   • ideally by at least 2 persons with an assessment of their inter-rater reliability;
   • should be made without knowledge of author's final conclusions, their identity or institution;
   • there exist guidelines for assessing quality;
   • ? rank studies by quality;
5. How have authors abstracted the information from individual studies?
   • data should be extracted without knowledge of result;
6. Have authors provided adequate details of the subjects included in the studies being analysed?
   • if marked dissimilarity in subjects then caution!
7. Have authors plotted their results?
   • eg. plot of log odds ratio with confidence intervals;
8. How have authors inspected the data for heterogeneity of outcome?
   • are the differences in indiv. trial outcomes greater than one could reasonably expect by chances alone, if so, then should the data be pooled at all?
   • assess with either O² or L'AbbJ plots;
9. How have authors calculated a summary estimate of the effect of intervention?
   • pooling must maintain integrity of individual studies;
   • contribution of each study is determined mainly by its study weight - usually 1/variance;
10. Have authors inspected data for evidence of publication bias?
    • “positive” studies are more likely to be submitted and accepted than “negative” studies → “anticonservative influence” & may depend in part on vested interests;
    • may be the most important bias affecting meta-analysis!
    • assess with “funnel plot” symmetry & base region;