Discussion
The key results are: (1) there are
differences in consultant neurologist diagnoses and investigative
approach in the general neurology outpatient setting; (2) whilst
differences in diagnosis are generally minor, a significant proportion
have potentially important management implications; (3) greater
diagnostic certainty is associated with higher diagnostic agreement and
a lower likelihood of arranging investigations; and (4) individualized
metrics can be produced representing different facets of clinical
practice. Consideration could be given to developing this system of peer
comparison as a tool to facilitate reflective practice at appraisal.
The case-mix encountered in this study by a newly qualified consultant
neurologist, working in a district general hospital, appears
representative of typical outpatient practice and current UK curriculum
content, with headache, sensorimotor disorders, blackouts and tremor
common problems. This differentiates our study from most others
published to date, in which unselected cases have been studied
infrequently. Previous literature on diagnostic consensus and
investigative approach has focused on highly specific issues. One
study1 assessed classification of dysarthria by 72
neurologists from patients’ speech recordings and found low agreement
(Cohen’s kappa 0.16-0.32). However, a different study6on clinical assessment of stroke found good inter-observer agreement in
diagnosis (91/98 cases, Cohen’s kappa=0.77). These conflicting data
suggest that disease group may be an important factor, and this was
supported by our study, in which balance disorders exhibited lowest
diagnostic agreement, followed by headache, sensory/motor disorders,
disorders of consciousness and tremor. Possible reasons for varying
inter-rater agreement between disease groups include greater
heterogeneity of disorders, fewer formalized diagnostic protocols, or
more overlap between different disorders with similar phenotypes. For
example, overlap in the clinical features of migraine and tension
headaches exists, with 58.4% of definite migraine patients having
tension-type symptoms and 68.1% of tension-headache patients reporting
migraine-type symptoms in one study7. Furthermore, in
this context, the primary aim of a consultant neurologist may be to rule
out secondary headaches. Despite not always agreeing on the primary
headache disorder, there were only three cases in which one consultant
suspected a secondary headache disorder in contradiction to the others
and none were found when investigations were performed, implying
agreement in decisions with implications for patient safety. Previous
research has also highlighted the difficulty in diagnosing balance
disorders; the sensation can be difficult to define, with numerous
possible causes, overlapping phenotypes and frequently inconclusive
investigations8. Previous studies9have found that a firm diagnosis may not be attainable in up to 20% of
cases.
Relationships between diagnostic confidence and experience have been
investigated in previous studies. In medically unexplained symptoms,
greater clinician experience was associated with increased confidence in
diagnosis amongst a mixed group of neurologists, cardiologists,
gastroenterologists and rheumatologists4. A comparison
of US and UK neurologists10 using a similar case-based
methodology to our study, found that more experienced neurologists
ordered fewer investigations. This study had a far greater number of
participants (705), but far fewer number of cases (3) than ours, so the
two studies provide complementary data on this issue. In contrast, the
GALATEA trial, an observational study into practice variability in
Alzheimer’s and cerebrovascular disease, found no relationship between
experience and investigative decisions across 107
physicians2. Whilst we lacked statistical power to
investigate relationships between experience and diagnostic confidence
in our study, we were able to evaluate associations between diagnostic
certainty, diagnostic peer-agreement and investigative approach, which
are novel data, and we consider may have value as an appraisal tool. All
UK clinicians undertake annual review and 5-yearly revalidation
procedures to monitor and maintain competencies. Individualized metrics
allow comparison of diagnostic consensus with peers, and could capture
more complex aspects of clinical practice. For example, the strength of
positive association between diagnostic certainty and diagnostic
agreement may reflect aspects of diagnostic judgement, and associations
between diagnostic certainty and investigative approach may indicate an
individual’s reliance on their clinical judgement, approach to resource
usage and management of uncertainty. Our data are supported by a
previous qualitative study11 of general practitioners
that found diagnostic uncertainty to be a positive determinant for
ordering tests in 22 physicians surveyed. Neurologists in our study were
sometimes uncertain. There is literature on teaching medical students
the management of uncertainty12 in an age when perhaps
patients, and potentially doctors too, may trust in technology more than
clinical judgement. Diagnostic uncertainty can result in
over-investigation13, which carries a risk of
incidental findings that can be detrimental. Management of uncertainty
may become especially important at transition to consultant level, with
increased responsibility, and has not traditionally been part of medical
school curricula. Further research into the influence of uncertainty on
clinicians’ decision-making may help develop tools to facilitate and
support recognition, management, discussion and documentation of this
normal aspect of clinical practice. This may be especially important in
the current era of Covid-19, when face-to-face contacts and access to
investigations are limited, potentially increasing diagnostic
uncertainty.
A limitation of our study was that follow-up data, to confirm veracity
of the diagnoses made, was often not available, and we recognize that
diagnostic peer-agreement is not synonymous with correct diagnosis.
However, this limitation accurately reflects the real-world general
neurology clinic setting in which the study was conducted; many
diagnoses in neurology are clinical, follow-up and investigations are
not always necessary or conclusive and, conversely, a strength of our
study is the external validity of investigating unselected consecutive
cases. A further limitation is that, whilst our methodology was designed
to simulate real-life practice, the raters not present in clinic with
the patients did not have the benefit of taking the history and
performing neurological examination themselves, and may have picked up
additional signs, or non-verbal cues and clues, which can be important
for diagnosis14. This potential bias between
theoretical and actual practice could be investigated by two or more
raters assessing the same patients and performing a reciprocal analysis,
with the factor of interest management differences between consultants
that met the patient or theoretically answered the questionnaire.
Despite these limitations, we propose that this methodology has
potential as an appraisal tool, with the advantages of producing
quantitative data derived from a directly relevant clinical setting and
enabling anonymized peer comparison of complex aspects of practice. Lack
of standardized assessments15, time
burden16 and subjectivity of current
assessment15 have been identified as problems with
current appraisal tools. Only 43% of physicians reported changing their
practice in response to their appraisal in one
study17, with another finding that 43% of physicians
felt that patient safety had not been improved by
revalidation18. New approaches based on real-world
practice might help address some of these areas of need, although would
have to be adapted to be feasible in a non-research setting. Collating
larger datasets on diagnostic and investigative approaches may also help
plan neurological services by helping direct resources appropriately and
enabling feedback loops to be developed between resource providers and
clinical practitioners.
In summary, there is variability in consultant neurologist practice,
which may influence management and can be captured by metrics reflecting
individual approaches. With refinement and consideration of the
limitations listed above, the methodology applied in this study may be
developed into new tools to facilitate reflection and appraisal, and
maintain the current high standards of neurological practice.