Using Respondent-Driven Sampling to Better Support Patients With ME
The Difficulties Associated with Diagnosing ME
Myalgic encephalomyelitis, otherwise referred to as ME, is a long-term condition, classified as a disorder of the nervous system (World Health Organization code G93.3). ME is a debilitating and complex medical condition characterized by disabling fatigue, and post-exertional malaise (PEM); where symptoms worsen after minor physical or mental exertion.
The illness has no clear mechanisms and no clinical biomarker for simple detection. This has led to disagreements over the validity of case definition criteria (the diagnostic criteria to define a condition) and no universally trusted approach for diagnosing the disease.
This presents two major challenges. Firstly, public health and welfare bodies have little at their disposal to determine when to provide support and disability statuses for patients with ME, and what types of interventions patients are likely to benefit from. Secondly, this uncertainty makes the diagnostic process long and complex. It can take between 4-6 years for a definitive conclusion to be reached for an individual patient.
As there is no clinical biomarker for the disease, existing diagnostic criteria have been developed mostly through the consensus of medical experts. This has generated concerns around the representativity of some patients’ experiences amongst the research and media communities. Further complicating this is the stigma associated with the condition, with some practitioners reluctant to diagnose it and some patients not wanting to be labelled as suffering with it.
Without the perspective of individuals living with ME, the hypothesis is that large numbers of sufferers go undiagnosed. In the US, undiagnosed patients have been estimated to be as high as 83% of those living with the disease.
The Fafo Research Foundation is a renowned Norwegian research institute that conducts social research both in Norway and internationally. For its latest project, Fafo wished to use its expertise to address the issues surrounding ME, in terms of both the identification of the disease and understanding it from the patients’ perspective.
A Different Approach to Identifying ME patients
One criterion that has become a critical point in diagnosing ME is PEM. In the Revised Canadian Clinical Case (CCC) Definition, PEM is mandatory making it a strong candidate to be the golden standard for diagnosis and research.
However, there is no sampling frame for this case definition. Researchers may be provided with a list of people with a G93.3 diagnosis but it is known that:
- Getting access to this type of personal data from patient registers takes time (assuming you get approval), due to ever-evolving GDPR. The use of health-register generated address lists has already had a knock-on effect to response rates.
- There is underdiagnosis of ME patients with G93.3, particularly against the socially disadvantaged.
- The diagnostic process is slow, presenting a bias against new cases.
- The diagnosis of ME is not always given in accordance with the CCC.
This is why the experts at Fafo decided to trial respondent-driven sampling (RDS) – to address these challenges, as it was developed to produce unbiased samples from populations without clear sampling frames.
RDS mimics a ‘random walk through a respondent’s social networks’. It combines snowball sampling, where respondents recruit participants, with mathematical modelling, to apply RDS estimators that offset the non-random method.
Although the natural assumption would be that such a method would introduce a sampling bias, studies have shown a convergence towards an ‘equilibrium’ as the network widens and becomes representative of the target population. In addition, RDS statistical tools generate estimators to counter bias derived from in-group recruitment and increased recruitment probabilities, resulting from having large social networks.
RDS also restricts access for the most active online patients, as people participate out of a social obligation to the acquaintance that recruited them.
To implement this chosen methodology Fafo needed a data creation partner who could help implement RDS online, to accurately identify participants, and provide the most effective survey instrument to maximize engagement. For this they partnered with Walr.
This research aimed to:
- Increase the knowledge of the illness’ prevalence.
- Identify socio-demographic factors affecting the probability of being diagnosed with WHO code G93.3.
- Document ME patients’ experiences to assess potential similarities or differences between patient groups meeting different diagnostic criteria.
Diagnosing ME in Participants
To identify ME in the survey participants, Fafo used the DePaul Symptom Questionnaire and the 36-Item Short-Form Health Survey (SF-36) – key tools within the Revised Canadian Clinical Case (CCC) Definition for ME. It also incorporated 4 validated algorithms specific to different case definition manuals.
Research estimates the revised CCC diagnostic tool to have a sensitivity of 98% and a specificity of 38% (acknowledging that false negatives are more likely than false positives).
The RDS Survey Process
To perform the RDS survey, Fafo targeted a broad sample population of individuals with a ‘fatigue illness’. The team purposely cast the net wider than individuals with ME, as inclusion errors (people who don’t belong in the survey) can be corrected, whereas exclusion errors cannot.
They began the process with 3-4 respondents – known as “seeds” – who then each recruited 4 more respondents, and so forth.
Walr configured the survey links so that Fafo would know who recruited who, without storing any personal data. This enabled the team to understand how respondents used their networks, which became essential to generating bias-correcting RDS estimators.
Applying RDS Estimators
To eliminate sample bias, RDS statistical tools were used to generate RDS estimators.
If your network was small, i.e. you did not know many individuals with fatigue illness, then your probability of being recruited into the sample was low. On this basis, your responses to the survey were given a higher estimator as you may have had a unique perspective that could otherwise get overlooked, e.g. men tend to have smaller networks so their experiences could be underrepresented. Those with a big network within the ME community had a much higher probability of being recruited, therefore their responses were given a lower estimator.
Moreover, RDS tools detect in-group recruitment. When respondents recruit others that are very similar to themselves, the estimator will correct for bias.
Supporting Survey Participants
Alongside the study, Fafo created a Facebook group that the participants could subscribe to, to receive more information about the research and where they could ask questions. This built a greater level of trust between the respondents and the researchers and helped clear up misunderstandings or negative feelings respondents may have had as a result of the process.
There were 2 main challenges Fafo needed to consider when implementing the study:
- Performing an RDS survey within this demographic was unchartered territory, whether it could work was a matter of debate. The ME population had long been seen as deeply polarized. If participants chose to recruit people only from the group they were part of, it would make it difficult to reach the equilibrium needed for a representative sample.
- This is a very traumatized community that have had negative experiences with researchers before. Fafo needed to build trust within this group and maintain open communication lines. The methodology needed to be clearly explained and the participants needed to know how their data was being used. Maintaining people’s anonymity and integrity was crucial.
How Did Walr Assist the Research Process?
For this study to be performed online it was necessary to implement a technical solution that facilitated the RDS methodology; seamlessly tracking each seed and who they recruited, whilst also delivering a great respondent experience. By using the Walr Platform, Fafo could accumulate the necessary data for a comprehensive mapping of the “walks” through participants’ networks.
“The method used to deploy RDS online was fundamental to the success of this research project. The ability to effectively sample patients, without them leaving the comfort of their own homes, was undoubtedly the star of the show. It would have been impossible to implement without the help of Walr’s infrastructure and expert support.”
Anne Kielland, Researcher at Fafo
This study was the largest RDS survey Fafo has conducted with a total of 660 participants. Around 1/3 of the participants were not members of the ME Association and equilibrium – the point at which a sample is deemed representative – was reached at around 250 participants on key questions.
The methodology used showed the conflict between sufferers portrayed in the media is likely exaggerated, as seeds indiscriminately recruited new participants from within the ME Association and outside.
Key preliminary results from the study:
- 72% of participants met the case definition set out by the CCC.
- 74% of those with the G93.3 diagnosis code also met the criteria set by the CCC.
- 65% who did not have the G93.3 diagnosis code also met the CCC.
Symptoms and other factors held constant:
- Younger people are more likely to be diagnosed with G93.3.
- Those with a higher education are more likely to receive the diagnosis.
- Participants whose onset of symptoms is 2 years or less have a lower probability of receiving the diagnosis.
- More women are diagnosed with ME than men. However, the long-held assumption that this was due to men avoiding the doctors, or not discussing their symptoms, was not supported by the research.
Conclusions and Significance
Firstly, the study provided a strong case for why the RDS methodology has a place in any researcher’s toolkit, when it comes to targeting difficult-to-reach audiences.
It provided a respondent base that could deliver critical insights for the ME community and those dedicated to improving its understanding and treatment:
- The method produced what may be the first reasonably unbiased social science sample of people meeting the CCC.
- It allowed an assessment of how meeting the CCC corresponded with having the G93.3 diagnosis, improving knowledge on over- and underdiagnosing.
- It compared patients based on stricter and wider criteria, in terms of their experiences with common services and interventions.
- It offered service providers useful tools when considering what patients may benefit from, in terms of interventions at hand.
- The recruitment pattern of the participants showed the community is not as polarized as traditionally thought, supporting the assumption that RDS was indeed an appropriate sampling approach in this population.
These insights can help form the basis of a strategy to more effectively target ME patients in the general population – a vital improvement given early diagnosis has been shown to lead to more effective treatment.
What’s Next for Fafo and Online RDS?
With the success of this project, Fafo is turning its sights to what could be an even harder-to-reach audience that could benefit from such a methodology.
Young individuals with neurodiversities, such as Attention deficit hyperactivity disorder (ADHD), Autism and Tourette Syndrome (TS), suffer similar diagnostic challenges and stigmatization as those with ME. This means there is still a lot to be learned from those living with the symptoms.
Fafo is partnering again with Walr, alongside a design agency, to develop a questionnaire that will assist respondents during participation. Special attention is being given to the look and feel of the survey and how these neurodivergences respond to certain stimuli.
While not required for the ME population, this study will also introduce another aspect of RDS methodology – recruitment through incentives. By using gamification techniques, Fafo hopes to make the survey encouraging and enjoyable, whilst avoiding elements that could trigger adverse reactions. User testing is under way and the partnership looks forward to presenting its next set of important insights for patient care in these underrepresented populations.
Book a demotoday.
Get in touch to see how the Walr Platform can support your business.