\documentclass{article}
\usepackage[affil-it]{authblk}
\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
\usepackage{url}
\usepackage{hyperref}
\hypersetup{colorlinks=false,pdfborder={0 0 0}}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
  \errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}

\usepackage[utf8]{inputenc}
\usepackage[english]{babel}




\begin{document}

\title{Quality of systematic review and meta-analysis abstracts in oncology journals}


\author{Chelsea Koller}
\affil{Affiliation not available}

  
\author{Sarah Khan}
\affil{Affiliation not available}
  
\author{Jonathan Holmes}
\affil{Affiliation not available}
  
\author{David Herrmann}
\affil{Affiliation not available}
  
\author{Michael Bibens}
\affil{Affiliation not available}
  
\author{A. Benjamin Chong}
\affil{Affiliation not available}
  
\author{Matt Vassar}
\affil{Affiliation not available}
  



\date{\today}

\maketitle




Abstract

Purpose:  The purpose of this study was to evaluate the quality of reporting in the abstracts of oncology systematic reviews using PRISMA guidelines for abstract writing.

Methods:  Oncology systematic reviews and meta-analyses from four journals - The Lancet Oncology, Clinical Cancer Research, Cancer Research, and Journal of Clinical Oncology - were selected using a PubMed search. The resulting 337 abstracts were sorted for eligibility and 182 were coded based on a standardized abstraction manual constructed from the PRISMA criteria.  Eligible systematic reviews were coded independently and later verified by a second coder, with disagreements handled by consensus.   One hundred eighty-two abstracts comprised the final sample.  

Results:  The number of included studies, information regarding main outcomes, and general interpretation of results were described in the majority of abstracts. In contrast, risk of bias or methodological quality appraisals, the strengths and limitations of evidence, funding sources, and registration information were rarely reported. By journal, the most notable difference was a higher percentage of funding sources reported in Lancet Oncology. No detectable upward trend was observed on mean abstract scores after publication of the PRISMA extension for abstracts.

Conclusion: Overall, the reporting of essential information in oncology systematic review and meta-analysis abstracts is suboptimal and could be greatly improved.
  
Keywords: Review, Systematic; Meta-Analysis; Cancer; Medical Oncology; Abstracting as Topic; Funding
  
  
  
  
  
  

\section{Introduction}

Scanning journal abstracts allows clinicians to quickly determine the relevance of a particular article to their clinical practice \cite{Fleming_2012}. The abstract should be written clearly and sufficiently detailed such that clinicians can decide whether to read on if the article is in hand or to download an electronic version for further reading \cite{Hopewell_2008}. A recent study found that users of biomedical literature that searched PubMed predominately viewed abstracts exclusively after reviewing titles returned from their searches. These abstract views were well over two times as likely as full-text views \cite{20157491}. However, despite the importance of abstracts to convey essential information to users of research, clear and comprehensive reporting of core study aspects remains an issue. In an effort to address concerns about the quality and clarity of abstract reporting in clinical trials, the Consolidated Standards of Reporting Trial (CONSORT) group developed a minimum set of essential information for inclusion in an abstract \cite{Hopewell_2008}. Since the CONSORT abstract extension was published in 2008, some improvement in abstract reporting has been noted but still remains an issue \cite{Can_2011}.

More recently, systematic reviews have played a growing role in decision making for clinic practice. While allowing biomedical literature users access to a higher quality of evidence, systematic reviews are still hampered by issues in the quality of abstract reporting \cite{23714302}. This prompted the release of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement that detailed a checklist of essential items to include in a systematic review abstract.

Since the extension of the PRISMA statement was published in 2013, no formal evaluation has been conducted on guideline adherence in medical journals. Only Kiriakou et. al.'s investigation of systematic review abstracts in oral implantology has been conducted to date \cite{24581704}. We, therefore, analyzed the extent to which systematic review authors reported this information in abstracts from a sample of leading oncology journals. We analyzed how well authors published in these journals adhered to the PRISMA extension guidelines for abstracts and whether this adherence had changed since the release of the PRISMA extension.
  
  
  

\section{Materials and Methods}

\subsection{Search Criteria}

Using the h5-Index from Google Scholar Metrics, we selected the 6 highest ranking oncology journals based on index scores. We searched PubMed using the following search string:(((((("Journal of clinical oncology : official journal of the American Society of Clinical Oncology"[Journal] OR "Nature reviews. Cancer"[Journal]) OR "Cancer research"[Journal]) OR "The Lancet. Oncology"[Journal]) OR "Clinical cancer research : an official journal of the American Association for Cancer Research"[Journal]) OR "Cancer cell"[Journal]) AND ("2007/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]) AND (((meta-analysis[Title/Abstract] OR meta-analysis[Publication Type]) OR systematic review[Title/Abstract]) AND ("2007/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]) AND (("2007/01/01"[PDAT] : "2015/12/31"[PDAT]) AND "humans"[MeSH Terms]). This search strategy was adapted from a previously established method that is sensitive to identifying systematic reviews and meta-analyses (Montori 2005). Searches were conducted on May 18 and May 26, 2015.

\subsection{Review selection and data extraction}

We used Covidence (covidence.org) to initially screen articles based on the title and abstract. To qualify as a systematic review, studies had to summarize evidence across multiple studies and provide information on the search strategy, such as search terms, databases, or inclusion/exclusion criteria. Meta-analyses were classified as quantitative syntheses of results across multiple studies \cite{25194857}. Two screeners independently reviewed the titles and abstracts of each citation and made a decision regarding its appropriateness for inclusion. Next, screeners held a meeting to revisit the citations in conflict and arrive at a final consensus. Following the screening process, full-text versions of included articles were obtained via EndNote. Full text screening was then completed and conflicts resolved by group consensus  excluding addition articles that did not meet requirements to be considered.  

\subsection{Coding Abstracts}

To ensure the accuracy of the coding process, an abstraction manual was developed and piloted prior to training coders. A training session was conducted to familiarize coders with the process, and subsets of studies from the screening process were jointly coded as a group. After training, each coder was given three new articles to code independently. These data were analyzed for inter-rater agreement by calculating the Cohen's kappa. As inter-rater agreement was acceptable (k=0.65; agreement=75 percent), each coder was assigned an equal number of articles for data abstraction. Elements from the abstraction manual are presented in Table 1. After the initial coding process, validation checks were conducted such that each coded element was verified by a second coder and a meeting was held to discuss disagreements and settle them by consensus. Figure 1 details the study selection process. Data from the final sample of 182 articles were analyzed using STATA 13.1 and are publicly available on Figshare ().\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/ChelsearTablepic/ChelsearTablepic}
\caption{{Table 1 Coding criteria used by researchers to assess abstract quality.%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/PRISMA-Flow-Diagram/PRISMA-Flow-Diagram}
\caption{{Figure 1 PRISMA flowchart.%
}}
\end{center}
\end{figure}

\section{Results}

\subsection{Search results}

The initial search in PubMed resulted in 337 articles. Using Covidence, we excluded 79 articles during the pre-screening of title and abstracts. During the coding process, there were 76 articled excluded (Figure 1), including 2 unable to be retrieved. The final analysis comprised 182 articles, resulting in 102 systematic reviews from the Journal of Clinical Oncology, 56 from Lancet Oncology, 18 from Clinical Care Research, and 1 from Cancer Research.  

\subsection{Abstract reporting across journals}

Across all journals, components of the PRISMA extension varied greatly in terms of reporting (Fig 1). The majority of systematic reviews or meta-analyses incorporated these terms into the title (149/182).  Additionally, the number of included studies (146/182), information regarding main outcomes (165/182), and general interpretation of results (174/182) were described in the majority of abstracts in our analysis.   Other components were reported less frequently.  Information related to the search, such as the databases (89/182) and dates searched (86/182), were reported in only about half of abstracts.   O.  

\subsection{Abstract reporting within journals}
  
In general, frequency of reporting of many components was consistent across journals (Table 2). (It should be noted that our search yielded only one study from Cancer Research, so we omitted this study from journal comparisons). While overall percentages on particular components across journals did vary, the reporting of certain items appeared with greater frequency across the 3 journals.  Perhaps the most notable difference was in funding.  The Lancet Oncology reported funding sources with the greatest frequency (20/58) which was much higher than what was reported in Journal of Clinical Oncology (5/106) and Clinical Cancer Research (0/17).  
  
\subsection{Abstract reporting over time}
  
Finally, we examined the mean score of each abstract by journal by year. For each abstract, one point was assigned if a particular component had been addressed (or 0 otherwise), and a summary score was calculated for each abstract.  Means of overall scores are plotted in Figure 3. As shown, there is not detectable trend across any journal.  The publication of the PRISMA extension for abstracts in 2013 did not produce a detectable upward trend in mean scores.\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=1.00\columnwidth]{figures/freguency/freguency}
\caption{{Figure 2 Reporting of PRISMA items across oncology journals.%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Results-3.2/Frequency-of-reporting-of-PRISMA-items}
\caption{{Table 2 Frequency of reporting of PRISMA items.%
}}
\end{center}
\end{figure}\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/Mean-Score1/Mean-Score1}
\caption{{Figure 3 Mean summary scores for \textit{The Lancet Oncology}, \textit{Journal of Clinical Oncology}, and \textit{Clinical Cancer Research}.%
}}
\end{center}
\end{figure}

\section{Discussion}
Our results suggest numerous points of weakness in the quality of abstracts from systematic reviews published in high impact factor oncology journals. Of the 17 items from the PRISMA extension for abstracts, no systematic review in our sample contained all abstract elements. On average, we found that SRs reported only 8 of the possible 17. Areas that were most lacking included languages included in the search process (12/182), registration number (0/182), funding source (25/182), and strengths and limitations (44/182). 

In comparing our results with previous studies, it is evident that abstract reporting from systematic reviews in oncology could use improvement. For example, many abstracts disregarded a description of search details. For systematic reviews, this information represents the thoroughness of the search in locating all relevant evidence, and this information should be easily accessible in a systematic review abstract. Beller et al. found that 90\% of abstracts included a search date and 60\% included the databases that were searched. Those findings are higher than the 47\% of abstracts reporting search date and the 49\% listing databases found in oncology abstracts.  Another important feature of systematic review methodology is the risk of bias assessment of the included primary studies. Only 19\% of oncology review abstracts mentioned a study quality or risk of bias assessment, which was less than half of the systematic reviews that included an assessment in Beller et al.'s article \cite{23714302}. This figure is also much lower than the 43\% found by Ochodo 2014 across other clinical areas \cite{Ochodo_2014}. 

One way to increase the quality of abstract reporting is to have authors present the information in a structured form.  Structured abstracts contain more information, are easier to read and search, and are welcomed by readers \cite{22453695}. Additionally, structured abstracts are favored by publishers as "over 50\% of the papers with more detailed abstracts... [are] subsequently published" \cite{23942740}. We propose that oncology journals consider a structured abstract mandate for new submissions that adheres to the PRISMA extension for abstracts. We believe more thorough abstract reporting would occur should journal editors adopt such policies.

Future research should continue examining the quality of abstract reporting in other clinical areas. Aside from this study, only Kiriakou 2014 has examined the quality of abstract reporting among systematic reviews and meta-analyses; therefore, questions remain regarding abstract reporting practices across medical disciplines \cite{24581704}.  Furthermore, our analysis indicated no detectable trend in abstract reporting after the PRISMA extension was published.  It is possible that due to the fairly recent publication date of this work, it may be too early to note substantial differences over time.  Future studies could determine whether more notable trends could be found.  

  
  
  
  
  
  
  
  
  
  
  
  

\section{Conclusions}

Concerning the PRISMA extension for abstract reporting, the abstracts of oncology systematic reviews and meta-analyses could be more clearly written to provide critical information to inform decision making.  In particular, these abstracts are lacking the inclusion of strengths and limitations, funding information, and risk of bias or methodological quality assessment.  Structured abstracts are known to include more information and should be considered for all study types by oncology journals. Furthermore, adoption of the PRISMA extension for abstracts bt journal editors would encourage a more thoughtful approach to abstract writing.

\selectlanguage{english}
\FloatBarrier
\bibliographystyle{plain}
\bibliography{bibliography/converted_to_latex.bib%
}

\end{CJK}\end{document}

