\documentclass{article}
\usepackage[affil-it]{authblk}
\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
\usepackage{url}
\usepackage{hyperref}
\hypersetup{colorlinks=false,pdfborder={0 0 0}}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}




\begin{document}

\title{Utilization of clinical trials registries in obstetrics and gynecology systematic reviews}


\author{Michael Bibens}
\affil{Affiliation not available}

  
\author{A. Benjamin Chong}
\affil{Affiliation not available}
  
\author{Matt Vassar}
\affil{Affiliation not available}
  



\date{\today}

\maketitle




\textbf{Abstract}
\\Objectives: We evaluated the use of clinical trials registries in published obstetrics and gynecological systematic reviews and meta-analyses.
\\Methods: A review of publications between January 1, 2007, and December 31, 2015, from six obstetrical and gynecological journals (\textit{Obstetrics \& Gynecology, Obstetrical \& Gynecological Survey, Human Reproduction Update, Gynecologic Oncology, British Journal of Obstetrics and Gynaecology, and American Journal of Obstetrics \& Gynecology}) was completed to identify eligible systematic reviews. All systematic reviews included after exclusions were independently reviewed to determine if clinical trials registries had been included as part of the search process.  Studies that reported using a trials registry were further examined to determine whether trial data was included in the analysis.
\\Results: Our initial search resulted in 292 articles, which was narrowed to 256 after exclusions. Of the 256 systematic reviews meeting our selection criteria, 47 utilized a clinical trials registry. Eleven of the 47 systematic reviews found unpublished data, and added the unpublished trial data into their results.
\\Conclusion: A majority of systematic reviews in clinical obstetrics and gynecology journals do not conduct searches of clinical trials registries or do not make use of data obtained from these searches.

  
  
  
  
  
  
  
  
  

\section{Introduction}

  

Systematic reviewers are known for using pre-specified, comprehensive searches to locate all relevant studies for inclusion. While thorough, this approach to searching may include an overrepresentation of published, statistically significant outcomes if reviewers search only popular databases such as MEDLINE or Embase. This occurrence is known as publication bias, which Dickersin defines as, "the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings" \cite{Dickersin_1990}.(p1385) To mitigate this issue, searches should attempt to use alternative information sources that contain grey literature and unpublished data.  

A recent survey of Cochrane systematic review authors examined their experiences when retrieving and using unpublished data for systematic reviews. Non-commercial clinical trials registries accounted for 6.3\% of sources reported \cite{Schroll_2013}. The infrequent use of trials registries reported by Cochrane reviewers is surprising since they offer promise in locating unpublished trial data. Furthermore, a number of precipitating events have made clinical trials registries a viable information source for systematic reviewers.

In 2005, the International Committee of Medical Journal Editors (ICMJE) created a mandate that all journals included in its vast membership network must require trialists to register prospective clinical trials involving human participants if that trial is to be published in their journal \cite{De_Angelis_2004}. The Food and Drug Administration Amendments Act (FDAAA), which was passed in 2007, requires all researchers conducting human clinical trials to register their trials with clinicaltrials.gov, the U.S. clinical trials registry ((cite) Food and Drug Administration Amendments Act of 2007. US Public Law 110-85; 21 USC 301, September 27, 2007.). These two events created an uptake in the number of clinical trials registrations as shown in Figure 1. Most recently, the World Health Organization (WHO) released a position statement in favor of registration of all clinical trials in a publically available, freely accessible, searchable trials registry (World Health Organization. WHO Statement on Public Disclosure of Clinical Trial Results. [cited August 6, 2015]; Available at http://www.who.int/ictrp/results/reporting/en/). The WHO has also issued a list of registries that meet such criteria.
  
  
  
  
  
  

In light of these events, questions remain regarding the use of clinical trials registries by systematic reviewers, and results from the few available studies addressing this issue show limited use of registries in these searches. For example, Sinnett et al. \cite{Sinnett_2015} found that only five of 78 systematic reviews (6.4\%) in clinical neurology searched clinical trials registries. Eight of 41 systematic reviews (19.5\%) in emergency medicine reported searching registries \cite{Keil_2014}. Even among high impact general medical journals, only 41 of 117 reviews (35\%) of used registry searches \cite{Jones_2014}. With recent findings that suggest significant publication bias in the reproductive health literature \cite{17437636}, we examined the use of clinical trials registries in systematic review searches in obstetrics and gynecology research. Of those systematic reviews making use of trials registries, we also investigated whether potential relevant unpublished trials were located from these searches and whether data from these unpublished trials were included in the systematic review.  
  
  
  

\section{Methods}

A PubMed search was conducted using the following search string: (((((("Obstetrical \& gynecological survey"[Journal] OR "Obstetrics and gynecology"[Journal]) OR "American journal of obstetrics and gynecology"[Journal]) OR "Gynecologic oncology"[Journal]) OR "Human reproduction update"[Journal]) OR "BJOG : an international journal of obstetrics and gynaecology"[Journal]) AND ("2007/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]) AND (((meta-analysis[Title/Abstract] OR meta-analysis[Publication Type]) OR systematic review[Title/Abstract]) AND ("2007/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]) AND (("2012/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]). Journals were selected based on the highest 2014 h5-index of Google Scholar. This search strategy was adapted from a previously published approach that is sensitive to identifying systematic reviews and meta-analyses \cite{15619601}. Searches were conducted on May 18 and May 26, 2015.

  
  
  
  
  
  
  

We used Covidence (covidence.org) to initially screen articles based on the title and abstract. To qualify as a systematic review, studies had to summarize evidence across multiple studies and provide information on the search strategy, such as search terms, databases, or inclusion/exclusion criteria. Meta-analyses were classified as quantitative syntheses of results across multiple studies \cite{25194857}. Two screeners independently categorized all articles and a follow-up meeting was held to discuss differences in screening. All disagreements were settled by consensus. Full-text articles were obtained via EndNote.  
  
  

To ensure accurate coding and clear directions, an abstraction manual was developed and piloted prior to training coders. A training session was conducted to familiarize coders with the process, and a subset of studies from the screening process was jointly coded. Coding elements included: title of article, author name(s), year of publication, whether clinical trial registries were searched and, if so, the names of the registries used. Each coder was assigned an equal number of articles to code. After the initial coding process, validation checks were conducted such that each coded element was verified by a second coder. After these checks were performed, coders met to discuss disagreements and settle them by consensus. Analysis of the final data was conducted using STATA 13.1. Data from this study are publicly available at \href{http://dx.doi.org/10.6084/m9.figshare.1491500}{http://dx.doi.org/10.6084/m9.figshare.1491500}.
  
  
  
  
  
  
  
  

As in previous studies,\cite{Sinnett_2015}\cite{Keil_2014}\cite{Jones_2014} we used the World Health Organization's International Clinical Trials Registry Platform (WHO ICTRP, List Version 2.1) \cite{Jones_2014}. Registries met specific criteria for quality, validity, content, accessibility, unambiguous identification, technical capacity, and administration/governance \cite{undefined}. Sources such as the Cochrane CENTRAL Registry of Controlled Trials were not included as clinical trials registries, but treated as a bibliographic database as CENTRAL features a selection of already published trials.
  
  
  

\section{Results}

  

Our search of PubMed yielded 292 articles, which was narrowed to 256 articles after exclusions (Fig. 2). As displayed in Fig. 3, we examined the use of clinical trials registries during the systematic review search process. Of the 256 articles, 47 (18.36\%) conducted a search in at least one clinical trials registry. The registries that were searched were clinicaltrials.gov (n = 44), the Chinese Clinical Trials Register (ChiCTR; n = 1), the International Clinical Trials Registry (n = 1), and the WHO Clinical Trials Registry (n = 1). Five of the articles that searched clinicaltrials.gov also searched the Australian New Zealand Clinical Trials Registry (ANZCTR). For the two articles that searched the International Clinical Trials Registry and the WHO Clinical Trials Registry, we determined that these articles referred to the WHO ICTRP.
  

Figure 3 also highlights that Obstetrics and Gynecology (n=36) was the only publication in our systematic review where more than 10 published studies searched clinical trials registries. In the other five publications that were analyzed, the mean number of studies that included a search of clinical trials registries was 4.4. 

Of the 47 studies that searched clinical trials registries, 36 did not make a clear indication whether or not they obtained or used any data from clinical trials registries. The remaining 11 systematic reviews obtained data from studies found on clinicaltrials.gov. However, no specific connections between the studies found in the trials registries and the data in the systematic reviews were ever established.
  
  
  
  

\section{Discussion}

Our results show that searches of clinical trials registries were not used in the majority of 256 obstetrical and gynecological systematic reviews examined. Clinical trials registries contain studies with unpublished data, and the inclusion of these studies is necessary to attain accurate answers to clinical questions. The exclusion of unpublished studies and data poses a greater risk of publication bias and likely affects the magnitude and direction of summary effect sizes. The extent to which publication bias produces inaccurate study results may impair the physician's ability to use results from systematic reviews to make clinical decisions.


Our analysis found that 47 of 256 (18\%) obstetrical and gynecological systematic reviews conducted a search of a clinical trials registry. This is a much higher percentage than was found in the neurological literature by Sinnett et al. \cite{Sinnett_2015} (6\%) and includes almost four times as many systematic reviews in the analysis. Our findings are similar to the results found in emergency medicine, which showed 19\% of studies searched trials registries \cite{Keil_2014}. Thirty-five percent of studies from high impact general medical journals reported searching trials registries, which roughly doubles the amount we found in the obstetrical and gynecological literature \cite{Jones_2014}. Overall, their use is still quite low.

In addition to increasing the use of clinical trial registry searches, there is also a need to assess publication bias in obstetrical and gynecological systematic reviews. By better understanding the prevalence of publication bias present in the literature, the consequences of not using clinical trials registry searches may be better understood. Furthermore, a reassessment of publication bias in systematic reviews that did not search clinical trials registries is also necessary to determine if there are noticeable differences in results.

Authors have many factors to consider before beginning a literature search. As shown in our results, however, searching clinical trials registries is not high on the priority list for most. Analyzing the frequency of clinical trial registry searches in medical literature is important, however, determining why authors of systematic reviews and meta-analyses are not using trials registries is equally valuable. Without addressing the barriers that deter authors from performing clinical trials registry searches, there will not be an increase in the number of searches conducted in systematic reviews.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/fig-3/fig-1}
\caption{{Fig. 1. Number of registered studies over time from clinicaltrials.gov.
**ICMJE mandate
+Passage of FDAAA%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/fig-1/fig-1}
\caption{{\textbf{Fig. 2. Selection process of articles for review \textit{PRIMSA Statement Flow Diagram}.}%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/fig-2/fig-2}
\caption{{\textbf{Fig. 3. Frequency of use of clinical trials registries in obstetrics and gynecology systematic review searches.}%
}}
\end{center}
\end{figure}

\selectlanguage{english}
\FloatBarrier
\bibliographystyle{plain}
\bibliography{bibliography/converted_to_latex.bib%
}

\end{CJK}\end{document}

