In moving from our traditional model of evidence based medicine (EBM) to Personalised Medicine (PM) and ensuring that greater participation and inclusion is built into our service designs, we are going to have to move away from the conventional statistical analysis to a new way of generating measures to identify best practice.
We live within budgets and have limited resources so delivering value and being cost effective, both with our finances, but also with our staff and environment is a challenging balancing act.
We know that our current mindset has led to bias and created systemic disadvantage, so we need to shift to expand our quantitative approach to include qualitative research methodology. We need to learn the skill of interpretation through an inclusive lens, understanding the limitations of our data sets and being prepared to adopt the principle of ‘unlearning’ that which has not worked.
Importantly, we are missing the process of looking back and ‘marking our homework’ so we can check that our assumptions and ensure our direction of travel is positive.
No doubt our ‘family GPs’ built their knowledge through true understanding of families over time. Our hospital staff used to be a permanent fixture and generated wisdom for the community. We have lost this capability, through early retirement, rapid career progression and staff turnover, short term outcomes and service evaluations with ‘in the moment’ decision making.
This shift from outcomes and meta-analysis to triangulation through mixed methodologies, using a diverse perspective to interpret and create insight which is reviewed over time as well as in the moment is our next step.
Our traditional model of evaluation is based upon: Input – Intervention – Outcome and we need to move to: Experience – Conversation – Understanding but what new measurements are needed to demonstrate success.
As we drown in information, how should evidence be generated and disseminated?
How can evidence generate evidence-based policy that is flexible to changing context?
How can we educate our front line workforce on the best options and be confident in using their intuition and judgement?
What does consumer led services look like in the eco-system of health and social care?
What are the current research methods that are available?
Healthcare research is a systematic inquiry intended to generate robust evidence however many of us, have limited experience in critically reviewing evidence, analysing statistics and are even less familiar with the field of qualitative research and its validity and we are not practiced at the skill of reviewing evidence with others to create understanding.
I have summarised the common qualitative methodologies, as healthcare professionals, we should become more familiar with all our models of research and understand the problems and how statistical analysis has led to misunderstandings.
The major types of qualitative research designs are:
- narrative research
- phenomenological research
- grounded theory research
- ethnographic research
- historical research
- case study research
The qualitative method of inquiry examines the ‘how’ and ‘why’ of decision making, rather than the ‘ ‘what,’where, ‘ and ‘when.’ Unlike quantitative methods, the objective of qualitative inquiry is to explore, narrate, and explain the phenomena making sense of complexity and does not test a hypothesis but discovers emerging themes.
Narrative research
Narrative research focuses on exploring the life of an individual telling their stories of individual experiences. Data collection include interviews, field notes, letters, photographs, diaries, and documents collected from one or more individuals. Data analysis involves the analysis of the stories or experiences and developing themes to gain greater insight.
Phenomenological research
Phenomenology defines the ‘essence’ of an individual’s experiences regarding a certain phenomenon. Data collection is through Interviews with individuals, examining documents and observations. Data analysis involves the researcher interpreting the phenomenon, based on their judgement, rather than simply describing it.
Grounded Theory Research
Grounded theory comes from the ability to induce a theory grounded in the reality of study participants. Data collection involves recording interviews from many individuals until data saturation. Data analysis includes analysing data through ‘open coding,’ ‘axial coding,’ and ‘selective coding.’ Open coding is the first level of abstraction, and refers to the creation of a broad initial range of categories, axial coding is the procedure of understanding connections between the open codes, whereas selective coding relates to the process of connecting the axial codes to formulate a theory. Results of the grounded theory analysis are supplemented with a visual representation of major constructs usually in the form of flow charts or framework diagrams. Quotations from the participants are used in a supportive capacity to substantiate the findings.
Ethnographic research
Ethnography is used for understanding culture-specific knowledge and behaviours. Ethnography focuses on narrating and interpreting the behaviours of a culture-sharing group. To understand the cultural patterns, researchers observe the individuals or group of individuals for a prolonged period of time. Ethnographers collect data by observation, interviews, audio-video records, and document reviews. A written report includes a detailed description of the culture sharing group.
Historical research
Historical research creates insights from the past and involves interpreting past events in the light of the present. The data is collected from primary and secondary sources such as diaries, first hand information, and writings. The secondary sources are textbooks, newspapers, second or third-hand accounts of historical events and medical/legal documents. The written report describes ‘what happened’, ‘how it happened’, ‘why it happened’, and its significance and implications to current clinical practice.
Case study research
Case study research focuses on the description and in-depth analysis of the case or issues illustrated by the case. Observations, one to one interviews and documents are used for collecting the data, and the analysis is done through the description of the case. From this, themes and cross-case themes are derived and a written case report constructed.
Lens of Philosophy
If our data, regardless of qualitative or quantitative origins, is only interpreted through the lens of a scientist, it is likely that we will continue to attempt to build rules and fail to take account of the complexity of the problem. Through the lens of philosophy, we can build on our concepts, explore possibilities, embed time and consider domains of spirituality, connection and allow other alternatives to be revealed.
Through understanding the patterns of health behaviours, illness experiences and context, we can design health interventions and develop healthcare theories which includes a person-centered way of discovering and uncovering thoughts and actions for us all.
Our current position
Evidence-based medicine (EBM) currently is defined as the optimal integration of the best research evidence, clinical expertise and patient’s unique values for clinical decision-making and for optimising patient care.
NICE has published evidence across:
- Conditions and Disease
- Health and Social Care Delivery
- Health Protection
- Lifestyle and Wellbeing
- Population Groups
- Settings
Although, NICE is increasingly widening the perspective of their stakeholder engagement and including qualitative research they still prioritise:
- Systematic Review and Meta-analysis
- Economic Evaluation in Health Care
- Quasi-experimental Evaluation of Health Care Programs and Policies
- Randomised evaluations
As healthcare professionals it is important we consider the following challenges of evidence based medicine.
Publication bias
Publications tend to publish statistically significant results which are positive proving the hypothesis which generally is that the intervention works. Many trials are never published and those with negative findings, even if significant are left unseen.
Small trials are often considered not worthy of publication as their results are not statistically relevant and yet often these ‘hint’ at important findings. An example includes the delay in adoption of thrombolytic treatment for acute myocardial infarction where as early as the 1950s, small trials showed reductions in mortality rates, but we had to wait for a trial which involved the enrolment of 48,000 patients before it became national best practice.
Poor quality research
Researchers may use the wrong methodologies, collate data incorrectly, use surrogate outcomes, misinterpret results and often came to unjustified conclusions. Under-representation and lack of reflection through fixed mindsets mean that misunderstanding in conclusions may be seen.
Discredited Trials
Discredited trails often continue to be cited as best practice as papers remain accessible. If a busy health practitioner does a quick ‘google’ search, the discredited paper may be the first to appear, with no identification of its flaws.
Under reporting of harms
Harm outcomes are often incompletely reported which leads to overestimation of efficacy and underestimation of safety.
Conflicts of interest
Conflicts of interests are widespread amongst academic institutions and researchers and associated with pro industry conclusions. Ghost authorship where there is a failure to declare conflict of interest or publish an advertisement with the appearance of a robust publication can mislead.
Trials stopped early for benefit
A significant number of trials stop earlier than planned due to apparent benefits that overestimate their true effectiveness. These trials receive great media attention and then affect clinical practice.
Statistical heterogeneity (how different we are)
As pharmacokinetic and pharmacodynamic mechanisms are fairly similar across all humans, patients appear to be relatively homogeneous (the same) in drug response. However, heterogeneity (differences) still occur, even if they are more likely to be due to the wider eco-system of social determinants of health and behaviours rather than variations in biochemical mechanisms.
When dealing with outcomes, EBM assumes that the “reality” of the drug or intervention response as statistically homogeneous (the same); so we all react in the same way thus an average response represents individuals well.
However we see heterogeneity (differences) in trials:
1) between-patient variability (the differences between patients)
2) patient-by-intervention interaction (the extent to which the intervention response differs between patients)
3) within-patient error (the variability that the same intervention given to the same person on different occasions may alter)
The more different we are, the more difficult it is to calculate the anticipated linear correlation between cause and effect and this leads to misinterpretation of the data.
Future research methods must find ways of accommodating clinical reality and that we may not behave the same, as currently the sad truth is that the more complex a situation is, the less evidence is available to treat them.
What are Outliers and why do we care about them?
The paradigm of EBM, through the use of statistics has created a world seen through averages.
The presence of outliers can lead to error in interpretation and tend to be removed through statistic analysis, with the consequence that systemic bias has formed amplifying the mean. Casual observation of the literature suggests that researchers rarely report outliers or seek to understand them. Given that we know that analysing our assumptions to ensure accurate interpretation is crucial, this is an unintended consequence which requires urgent review.
In the future, the entire Bells Curve should be analysed, with the understanding that the outlier rather than being a ‘contaminant’ to ignore should be considered as a source of insight.
Outliers from data errors.
Outliers are often caused by human error, such as errors in data collection, recording, or entry.
Outliers from intentional or motivated mis-reporting.
There are times when subjects purposefully report incorrect data to researchers, often from the act of trying to please rather than sabotage. Data is often sensitive (e.g., teenagers reporting drug or alcohol use or sexual behaviours) and therefore reporting may not be accurate. Motivated over-reporting can occur when the variable in question is socially desirable such as income, or educational attainment and the Placebo effect and Hawthorne effect are well studied.
Outliers from sampling error.
Another cause of outliers or is sampling. It is possible that subjects are inadvertently drawn from a different population than the rest of the sample such as studying nurses and inadvertently, a paramedic is interviewed but the study is about nursing views.
Faulty or non-calibrated equipment is another common cause of outliers.
Finally, it is possible that an outlier can come from the population being sampled legitimately through random chance and this therefore is a valid subject to understand!!!!
When researchers in Africa discovered that some women were living with HIV for years without treatment, these rare cases formed a source as inspiration for inquiry: what makes these women different or unique, and what can we learn from them?
So what are we going to measure to ensure we have services that meet the needs of our populations.
What do people probably need to know?
- Was the intervention accessible?
- Was it delivered to the standards required?
- Did it make a difference to the person?
What do those responsible for population health need to know?
- What does the whole system look like to those who use services?
- From this jigsaw, is it accessible by all?
- Which elements are measurable and how can these be measured?
- What difference does it make to the population and is it inclusive?
Does looking back over time give greater insight?
We have an exciting opportunity with integrated care systems to provide a systems approach to health and social care and to relook at our key performance indicators, national audits and other benchmarking tools to ensure that we are not reinforcing bias and review our existing evidence base with the eye of inclusion.
The transition from ‘outcomes that matter to organisations’ to ‘outcomes that matter to patients’ is an important step forward.
- Our demographic data needs to more than protected characteristics reflecting heritage, culture, geography and socio-economic status.
- Our goals should align with our values creating measures for our organisations and communities.
- Resource utilisation needs measures which demonstrate whole system utilisation.
- Patient experiences of shared decision-making and narratives need to be recorded.
- We need to understand how our patients, staff and organisations feel and change from the current position of stress, fear and failure to feeling safe, trusted and motivated.
- We should analyse our insight using multiple perspectives creating thought leadership which stimulates innovation providing a catalyst of change.
Dr Gethin Rees has a mixed methods research project through which I will enjoy learning about qualitative research and building a new skill set. https://research.ncl.ac.uk/equivalence-in-custody-healthcare/
They are using:
- Ethnographic approach – fly on the wall to observe interactions
- Semi structured interviews to deep dive and explore
- Review of risk assessments and consideration of continuity of care and information sharing
I am also working alongside Dr Steven Suckling who is introducing philosophy into the review of health and care services. NHS Staffing Crisis and The Art of Life – Maslow Foundation (wordpress.com).
Triangulation of results should form the new gold standard, blending quantitative and qualitative research with a lens of science and philosophy. I would propose that the current landscape, rather than being the Gold Standard, should be considered the minimum standard and we can build on this to form a New Paradigm which includes co-creation, distributed wisdom and situational awareness.
Although this is a long video, it challenges our approach introducing philosophy, new language and constructs. I found this a motivating video that shows there is another way providing hope and inspiration for the future.
