Your Member Data is a Dumpster Fire. Now What?

Matt Kerr • March 25, 2025

By Matt Kerr, IFANR Research Director

At the Institute for Association and Nonprofit Research (IFANR), we are frequently putting out dumpster fires. It is why associations come to us. But some dumpster fires are more preventable than others, and one such case is with a member database.

First, let’s back up and look at what goes into a questionnaire. In addition to asking relevant questions in a meaningful manner, questionnaires might also seek out basic demographic information about the respondent, e.g., age, gender, race/ethnicity, or employment. Knowing this information allows an association to address the needs of its different constituencies.

Sometimes the client will reject adding these questions, insisting that their own database contains this information and that it can easily be appended to the mailing list used to send out survey invitations. Adding these questions to the questionnaire only makes the survey longer. It’s a good idea, but sometimes backfires.

The demographic data is often self-reported by the member, and therein lies the problem. People simply make mistakes when entering their own data. How many times have you mistyped your email address? Or never updated your address when you moved? A former co-worker told me that she incorrectly listed herself as “male” in her client’s own database. This is why we ask these questions again – to give respondents a chance to set the record straight, at least for this one survey.

That said, there are times when the respondent is wrong, but we need to let them think they are correct. A common example of this is when asking membership status. Invariably, a percentage of respondents will indicate that they are current members, when the client’s own records reliably report that they have not been members in several years. We defer to the respondent in these cases. We don’t want to follow up with a question asking someone why they didn’t renew their membership when they think they are still a member. It’s not a perfect solution, but it avoids upsetting the respondent and provides valuable feedback to the client.

And last, but not least, the client can make mistakes with their database. I once had a client that sent me three membership lists – one for current members, another for members, and the last for “never” members. Unfortunately, some appeared on more than one list, in some instances all three lists. The list was supposed to represent a specific moment in time (i.e., the date on which the lists were pulled), so it is unclear how someone could appear on more than one list at the same time. But there it was, and we had a dumpster fire raging out of control.

So that’s why we try to ask these questions in the survey. The answers allow us to resolve a lot of problems before it is too late to do anything about them. And IFANR can help your organization put out your database dumpster fires. Figuratively speaking, of course. We don’t handle real fires…yet.

< Older Post

Newer Post >

The Art of the Qualitative Interview

By Joseph Bates • April 21, 2025

There is an art to conducting a good interview. Regardless of the purpose, be that gaining insight on a new app or collecting feedback for a strategic planning meeting, an interviewer usually has about 30 minutes to make a participant feel comfortable, trusting, and open to sharing their personal thoughts and opinions. This can be a challenging environment for the interviewer to create, so here are the top 3 tips that the IFANR team recommends when conducing qualitative interviews! Begin the interview with a brief introductory conversation so that you and the participant can learn a little bit about each other. While the participant is introducing themselves, listen for any commonalities that the two of you share. The interviewer should be prepared to share a little bit about themselves as well. This brief conversation builds trust between the interviewer and the participant, especially if you happen to have something in common! Remember to ask follow-up questions that allow the participant room to expand on their answer, such as “Can you tell me more about that?” or “Tell me a little more about why you feel that way?” Asking these follow up questions might elicit a more complete and detailed answer than your original question! After reading many books and articles on how to effectively conduct interviews (and after conducting hundreds of interviews ourselves!) we remember this…a strong interviewer knows that we are there to bear witness to the participant’s story, to care about what they have to say, to suspend judgement, and to hold surfacing emotions with respect (Atkinson, 1998). Happy interviewing! References Atkinson, R. (1998). The life story interview. Sage.

Does Data Matter? - What Goes For Baseball, Goes For Associations

By Matt Kerr • April 14, 2025

In 2003, Michael Lewis published Moneyball: The Art of Winning an Unfair Game. Lewis examined how Billy Beane, the general manager of the Oakland Athletics baseball team, utilized an advanced analytical approach to maximize the talents of his players while still maintaining one of the lowest payrolls in baseball. Using this approach, Oakland performed above expectations and made several postseason appearances. What do you really know? Beane’s analytical approach was controversial. For over a century, baseball executives had built rosters and coaches had made in-game decisions using experience, hunches, and a hodgepodge of statistics. Advanced analytics demonstrated that this approach was often misguided. Today, organizations win by building teams around relevant data, not hunches. Market research tackles association issues in much the same way. How does your association know what it claims to know? Is it experience? Is it from talking to enthusiastic members at the annual conference? Maybe board members with their own pet interests? When you use any of these as your baseline, you are playing your hunches. Market research gives you real data from which to make decisions. Sometimes, your hunches are correct. Many times, they are not. Research is not execution. Hall of Famer Yogi Berra once said “Baseball is 90 percent mental. The other half is physical.” Despite Berra’s questionable math, his point is nonetheless valid. It is not enough to know what to do; one must execute to be successful. Knowing that an opposing batter will always swing at a curveball in the dirt is useless information if your pitcher can’t throw a curveball. Likewise, market research is not often useful on its own. A successful market research study will provide associations with the foundation from which to craft an effective strategy forward. A good research project tells a story. Baseball purists will sometimes maintain a scorecard (“keep score”) when attending baseball games. A holdover from the days before state-of-the-art video boards, a scorecard allows a fan to keep track of what is happening on the field. Part code, part art form, a good scorecard is not a highlight reel, but rather tells the complete story of the game. Baseball fans know that Bill Mazeroski hit a walk-off homer in the bottom of the ninth to win the 1960 World Series for the Pittsburgh Pirates. But do you know why the game was tied before that? The scorecard would tell you why. Good market research endeavors to tell the complete story, too. As an association, you may know parts of the story. You may even have empirical data to back up those parts of the story. But do you know the entire story? IFANR can help you uncover the many layers that will tell a more complete story. Doing more with less. Just like Billy Beane and the Oakland Athletics, associations must increasingly do more with less. Research allows associations to pinpoint actual problems, not perceived ones, and to focus attention on them. Don’t be let a curveball surprise you when you were expecting a fastball.

How to Maximize Survey Response Rates - Composing an Effective Email

By Joseph Bates • April 7, 2025

The Institute for Association and Nonprofit Research (IFANR) has dozens of years of experience sending survey email invitations. We have perfected our methodology for achieving the best response rates possible. Below are the steps you should follow when creating a survey email invitation. HINT: Do everything you can to make the survey email invitation look as if it is coming from a real person. 1) NEVER use html. 2) ALWAYS have the email invitation appear to come from someone respondents recognize at your organization. Don't use your generic marketing email template or account. 3) Use the sender’s actual email signature block. Again, you need to make this appear to come from someone real. 4) Keep the email SHORT. The FIRST sentence should explain that you are conducting a survey and need the individual to respond. 5) Always personalize the email greeting, such as "Hi Joe"...don't say "Dear Joe" if that is not how the person from whom the email is coming would normally start an email. NEVER say "Dear Member"...as that is too generic. 6) The only link that should be in the email is the link to the survey, and, of course, the opt-out link at the VERY bottom of the email. 7) Send no more than two follow-up reminders that are targeted ONLY to those who haven't completed the survey. 8) Keep the survey in the field for about 7 business days and stagger the reminder emails about two or three days apart. PUT THE DEADLINE IN THE LAST EMAIL INVITE ONLY. 9) Most importantly, make sure to include the following in your subject line: "Please Respond" or "Participation Requested" along with the title of your survey. You need this call to action.