Cyclogram, Facebook, GMCLA and Respondent Driven Sampling

Respondent-driven sampling (RDS) is a referral-based study participation model designed specifically to overcome the problem of sample bias in hard-to-reach populations. Study participants recruit their friends. Recruiters keep track of who recruits whom. 

RDS seems to be a natural fit for the viral marketing models we see on the Internet. However, results have so far been mixed. Referral processes have been complicated, and participation has been low.

With the Gay Men’s Chorus of Los Angeles, Cyclogram is testing a new RDS format using a specially designed Facebook application that controls and tracks the referral process to produce a statistical analysis that controls for recruitment biases. 

Respondent Driven Sampling isn’t something you read about everyday.  When we read about research studies in the popular press, journalists cover the sexy part: the ultimate findings. “Mitt Romney has declined in the polls.” “Lipitor shows significant decreases in heart attack and stroke.” It’s uncommon for a journalist to talk about the science behind how researchers recruited participants. Maybe they mention the confidence intervals: “The poll results are within the margin of error.” Maybe at the very end they say, “The sample size is small.” But behind the scenes, researchers spend considerable time thinking about who belongs in the sample and how suitable participants will be identified and recruited.  

For pollsters measuring public opinion and forecasting election results, a randomized sample consists of a carefully considered random-digit dial of likely voters or of intercepts at polling stations (what have become known as “exit polls”).  Likely voters are relatively easy to find. Even those techniques have their detractors, and they sometimes fail miserably when compared to the actual election results.  

If it is difficult to build a sample from hundreds of millions of likely voters, it is even more difficult to find sufficient numbers of suitable participants from smaller, hard-to-reach groups. We have a pretty good idea of the number of voters. What’s the total number of jazz musicians in the United States? We generally know where we can find voters. Where do you recruit people who have diabetes? Although random-dial surveys are becoming more complicated as people increasingly drop landlines for cell phones, pollsters can still have some confidence that they are reaching a representative sample when they make thousands of random telephone calls. Are the gay men you find at the Abbey in West Hollywood on a Saturday representative of all gay men in the United States? In Los Angeles? In West Hollywood? Are they even representative of men who go to the Abbey on other days of the week? 

Because of these challenges, researchers looking at very specific populations often describe their participant pools as representing “a sample of convenience,” which means little more than that the sexy study findings apply only to that specific sample—they can’t be generalized to a wider group of people. So that statement that all men who have sex with men do xyz? Researchers can only say with confidence that it applies to their specific sample, which may be recruited entirely from men leaving the Abbey on a Saturday night.