Journal of Clinical Epidemiology
Journal of Clinical Epidemiology
5 years ago

Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy

To determine whether disagreements among multiple data sources affect systematic reviews of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and non-public sources (clinical study reports [CSRs] and individual participant data [IPD]). Results We found 21 gabapentin RCTs (74 reports, six IPD) and seven quetiapine RCTs (50 reports, one IPD); most were reported in journal articles (18/21 [86%] and 6/7 [86%], respectively). When available, CSRs contained the most trial design and risk of bias information. CSRs and IPD contained the most results. For the outcome domains “pain intensity” (gabapentin) and “depression” (quetiapine), we found single trials with 68 and 98 different meta-analyzable results, respectively; by purposefully selecting one meta-analyzable result for each RCT, we could change the overall result for pain intensity from effective (standardized mean difference [SMD]=-0.45; 95%CI -0.63 to -0.27) to ineffective (SMD=-0.06; 95%CI -0.24 to 0.12). We could change the effect for depression from a medium effect (SMD=-0.55; 95%CI -0.85 to -0.25) to a small effect (SMD=-0.26; 95%CI -0.41 to -0.1). Conclusions Disagreements across data sources affect the effect size, statistical significance, and interpretation of trials and meta-analyses.

Publisher URL: www.sciencedirect.com/science

DOI: S0895435617307217

You might also like
Discover & Discuss Important Research

Keeping up-to-date with research can feel impossible, with papers being published faster than you'll ever be able to read them. That's where Researcher comes in: we're simplifying discovery and making important discussions happen. With over 19,000 sources, including peer-reviewed journals, preprints, blogs, universities, podcasts and Live events across 10 research areas, you'll never miss what's important to you. It's like social media, but better. Oh, and we should mention - it's free.

  • Download from Google Play
  • Download from App Store
  • Download from AppInChina

Researcher displays publicly available abstracts and doesn’t host any full article content. If the content is open access, we will direct clicks from the abstracts to the publisher website and display the PDF copy on our platform. Clicks to view the full text will be directed to the publisher website, where only users with subscriptions or access through their institution are able to view the full article.