Blog Layout

When Enough is Enough: Reaching the Saturation Point

Megan Deutschman • August 11, 2022

By Dr. Megan Deutschman, IFANR Research Manager

As part of our research methodology, IFANR conducts a number of one-on-one, in-depth interviews for our clients. Inevitably at some point in our partnership, our clients ask how many in-depth interviews are “enough” to get a reliable answer to the questions we are investigating. The answer is – it depends.

 

In qualitative research, we know we have conducted enough interviews when we reach the saturation point. Simply put, the saturation point is the point at which the data becomes redundant, and interviewee’s answers to our questions do not yield any new information (Glaser & Strauss, 2017; Sebele-Mpofu, 2020). For the type of research we conduct at IFANR (strategic planning, membership reviews, sponsored research programs, etc.) the saturation point is typically reached at around 5 to 10 interviews per member segment. For example, on a recent project I spoke with members of an agricultural association about issues in their industry. After about the 8th or 9th interview, I noticed most interviewees discussed “supply chain problems, transportation issues, threats to trade, and the political environment” when discussing industry challenges. This is when I knew I had reached the saturation point.

 

This being said, the number of interviews needed to reach saturation is highly contextual and very dependent on your research question(s). In my dissertation, for example, I conducted research on complicated social problems and their intersection with the education system, so it took almost 40 interviews to reach the saturation point. This large volume of interviews was due to the fact that the topic I had chosen to study was highly complex, and I was also using a particular methodology that required a substantial number of interviews over time.

 

It is because of this variability in saturation that IFANR works very closely on methodological design with our clients, thus ensuring that we conduct the proper number of interviews to gain a clear picture of the research problem without wasting precious time and money.

 

References

Glaser, B. G., & Strauss, A. L. (2017). The discovery of grounded theory: Strategies for qualitative research. Routledge.


Sebele-Mpofu, F. Y. (2020). Saturation controversy in qualitative research: Complexities and underlying assumptions. A literature review. Cogent Social Sciences, 6(1).

Your Member Data is a Dumpster Fire. Now What?

By Matt Kerr • September 1, 2022
By Matt Kerr , IFANR Research Director

What to do When an Interview Goes Wrong

By Megan Deutschman • August 25, 2022
By Dr. Megan Deutschman , IFANR Research Manager

Survey Writing: The Questionnaire as a Story

By Matt Kerr • August 18, 2022
By Matt Kerr , IFANR Research Director

The Problem with Long Lists in Survey Questionnaires (and The Simpsons)

By Matt Kerr • August 4, 2022
By Matt Kerr , IFANR Research Director

The Story Behind the Numbers: Mixed-Methods Research

July 28, 2022
By Dr. Megan Deutschman , IFANR Research Manager

Frequent Questionnaire Mistakes

July 13, 2022
By Matt Kerr IFANR Research Director

The Value of Qualitative Research

July 7, 2022
By Dr. Megan Deutschman , IFANR Research Manager

Generating Non-Dues Revenue Through a Sponsored Research Program

By jturner • April 12, 2022
When Joe Bates, president of IFANR, was vice president of research and strategic initiatives at the Global Business Travel Association, he helped grow the sponsored research program from $100,000 in revenue in 2010 to $1.5 million in 2015. This huge increase in revenue allowed the association to do the following: 1) Published research reports that benefited all members, 2) Generated thought leadership for the association, 3) Earned massive media attention for the industry, 4) Funded other strategic research projects for the organization itself, and most importantly 5) Generated non-dues revenue for the association. The Institute for Association and Nonprofit Research (IFANR) can help your association create a sponsored research program to generate non-dues revenue as well as realize other benefits. Here’s a short “How To” list to get your sponsored research program started: 1. All research sponsors receive recognition on an association “Research Donor Wall” at the appropriate level (Silver, Gold, Platinum, Diamond). The Donor Wall is listed in various places, such as the association’s website, on-site at various conferences, in email blasts, etc. 2. Research sponsors are invited to contribute ideas and questions to studies, however, the association has the final say over all questions to ensure third-party objectivity and credibility. 3. Organizations receive prominent attribution as the research sponsor in the final report. 4. At the completion of each research study, the association issues a press release to the media announcing the report’s key findings. The research sponsor receives attribution in the press release and the opportunity to provide a quote. 5. The research reports are available to all members on the website. Some research reports may require purchase by members who do not chose to participate in the research project. Most research reports are available for purchase by non-members, although some research may be available to members only. 6. Sponsors may be allowed to send the report to their customers and post the report on their website as long as it resides behind a firewall that only customers are allowed to access. 7. Certain research studies may be provided to anyone free of charge and can be posted on the sponsor website for lead-generation and/or other marketing purposes. 8. At the completion of the research study, the association schedules a webinar for members and non-members to present the key findings of the study. The research sponsor receives attribution at the beginning and end of the webinar. In addition, at the beginning of the webinar, the research sponsor is allowed a two-minute “thank you” spot. The sponsor MUST NOT engage in any selling. Typically, sponsors receive the email list of participants for follow-up marketing efforts. 9. For major sponsors, the association presents the results of the research at a conference. The research sponsor introduces the presentation (two-minutes) and receives attribution as the research sponsor. Again, the sponsor MUST NOT engage in selling at this time. 10. Depending upon the level of sponsorship, the association may wish to grant the sponsor exclusive rights to create derivative materials based on the research.
man and woman shaking hands

The Art of the Qualitative Interview

March 31, 2022
There is an art to conducting a good interview. Regardless of the purpose, be that gaining insight on a new app or collecting feedback for a strategic planning meeting, an interviewer usually has about 30 minutes to make a participant feel comfortable, trusting, and open to sharing their personal thoughts and opinions. This can be a challenging environment for the interviewer to create, so here are the top 3 tips that the IFANR team recommends when conducing qualitative interviews! Begin the interview with a brief introductory conversation so that you and the participant can learn a little bit about each other. While the participant is introducing themselves, listen for any commonalities that the two of you share. The interviewer should be prepared to share a little bit about themselves as well. This brief conversation builds trust between the interviewer and the participant, especially if you happen to have something in common! Remember to ask follow-up questions that allow the participant room to expand on their answer, such as “Can you tell me more about that?” or “Tell me a little more about why you feel that way?” Asking these follow up questions might elicit a more complete and detailed answer than your original question! After reading many books and articles on how to effectively conduct interviews (and after conducting hundreds of interviews ourselves!) we remember this…a strong interviewer knows that we are there to bear witness to the participant’s story, to care about what they have to say, to suspend judgement, and to hold surfacing emotions with respect (Atkinson, 1998). Happy interviewing! References Atkinson, R. (1998). The life story interview. Sage.
baseball on plate

Does Data Matter? - What Goes For Baseball, Goes For Associations

March 29, 2022
In 2003, Michael Lewis published Moneyball: The Art of Winning an Unfair Game. Lewis examined how Billy Beane, the general manager of the Oakland Athletics baseball team, utilized an advanced analytical approach to maximize the talents of his players while still maintaining one of the lowest payrolls in baseball. Using this approach, Oakland performed above expectations and made several postseason appearances. What do you really know? Beane’s analytical approach was controversial. For over a century, baseball executives had built rosters and coaches had made in-game decisions using experience, hunches, and a hodgepodge of statistics. Advanced analytics demonstrated that this approach was often misguided. Today, organizations win by building teams around relevant data, not hunches. Market research tackles association issues in much the same way. How does your association know what it claims to know? Is it experience? Is it from talking to enthusiastic members at the annual conference? Maybe board members with their own pet interests? When you use any of these as your baseline, you are playing your hunches. Market research gives you real data from which to make decisions. Sometimes, your hunches are correct. Many times, they are not. Research is not execution. Hall of Famer Yogi Berra once said “Baseball is 90 percent mental. The other half is physical.” Despite Berra’s questionable math, his point is nonetheless valid. It is not enough to know what to do; one must execute to be successful. Knowing that an opposing batter will always swing at a curveball in the dirt is useless information if your pitcher can’t throw a curveball. Likewise, market research is not often useful on its own. A successful market research study will provide associations with the foundation from which to craft an effective strategy forward. A good research project tells a story. Baseball purists will sometimes maintain a scorecard (“keep score”) when attending baseball games. A holdover from the days before state-of-the-art video boards, a scorecard allows a fan to keep track of what is happening on the field. Part code, part art form, a good scorecard is not a highlight reel, but rather tells the complete story of the game. Baseball fans know that Bill Mazeroski hit a walk-off homer in the bottom of the ninth to win the 1960 World Series for the Pittsburgh Pirates. But do you know why the game was tied before that? The scorecard would tell you why. Good market research endeavors to tell the complete story, too. As an association, you may know parts of the story. You may even have empirical data to back up those parts of the story. But do you know the entire story? IFANR can help you uncover the many layers that will tell a more complete story. Doing more with less. Just like Billy Beane and the Oakland Athletics, associations must increasingly do more with less. Research allows associations to pinpoint actual problems, not perceived ones, and to focus attention on them. Don’t be let a curveball surprise you when you were expecting a fastball.
jbates@ifassociationresearch.com
Share by: