Should associations do their own qualitative research?
Even in our wildly data-driven world in love with quantitative data, qualitative research remains an unparalleled method of gathering evidence on the environment and performance of an association.
Since it’s really just talking to people, and because the investment can be fairly significant if done by a third party, it’s something associations will often take in-house, done either by staff or by volunteers. There are certainly advantages to that, including a more nuanced and intimate knowledge of the association by the people directly involved in it.
However, this decision should be made with a clear-eyed view of the trade-offs. Here are some specific advantages to using a third party that may not be immediately evident.
They’ll talk to us
Our promises of confidentiality are more credible. Even though they know who my client is (and don’t know who I am), people do seem to trust the arms-length aspect of the interview.
They also might be more curious to hear what a third party might ask them, so they might be more likely to agree to the interview. Or, they don’t feel they are tipping their hand to you by agreeing to do the interview at all.
They’ll tell us the truth
We’ll get a less-polished story. They know this is likely the only interaction they’ll have with us so they don’t feel the need to be terribly polite. We don’t work for the association, so we’re seen as more neutral on hot-button political issues.
It won’t just be a report card
We aren’t so directly connected to the association in people’s minds. This means that we can ask them about their priorities and needs, and people will talk about them longer without reference to the association specifically. This is enormously valuable when you’re doing strategic research which need to look beyond the current association’s boundaries and practices.
We’re probably better at it
External consultants generally (if they know what they’re doing) have some obvious advantages over association staff or volunteers. Often, we’ve got more experience and skill in setting up research processes and analyzing results. In a given year, even a very small firm may conduct hundreds of qualitative interviews and scores of focus groups.
(What will they tell me that they might not tell you?)
Here are a few things I hear in qualitative interviews – which I’m not sure would have been shared so readily with an association insider:
- They’ve got lots of alternatives
- They feel stuck with you.
- What you think they want isn’t what they want.
- They just aren’t that into you.
- They just don’t care about the bee that board member’s got in their bonnet
- You’re annoying them in specific ways.
- They’re mad at you.
- They’ve barely heard of you.
Yes, but we just can’t find the budget
So, if you really don’t feel you can round up the investment for a third party to do that research for you, but you still want the outputs, how can you have some success with a DIY qualitative research process?
- Identify how the research will be used, and let the interviewees know that as well. (Note: yup, you have now created an expectation from them that they will hear about some of the upshot.)
- Sit back and listen. No defending, no explaining, no arguing, no debate.
- Limit your questions. One or two open questions is often all you need to get someone really talking. It’s tempting to load a lot of questions in, but resist it. Follow up on interesting ideas if there’s time.
- Ask if there’s anything you didn’t ask that you should have.
- Thank those who gave their time for an interview.
- Draw some meta conclusions. Are interviews hard to schedule? Are they short? Is praise lukewarm? Do they seem hesitant to speak to certain issues?
- Perform a gap analysis. Think about what a highly positive response might be, in a perfect world; how does that compare with what you’re hearing? What about the difference between your results and what you think a third party would have heard? (Although caution: associations do not always guess correctly on this one.)
- Set up a clear methodology for capturing the results of the research. This is a particularly crucial tool when you have multiple people conducting the interviews.
Getting real value out of your work (or investment)
And the step that is far too often neglected in both internal and third-party research: analyze the results. What do they mean? What new ideas did they raise? What did they validate? What did they challenge? What are you going to do with what you now know?
That’s another way that third-parties can help. We’re skilled at taking the masses of information generated by qualitative research processes and analyzing them systematically, turning them into clear insights. Coming in from the outside, we don’t have pet projects or sacred cows.
If you’d like to discuss how we can help you use qualitative research (or quantitative, for that matter) to make better strategic decisions, please get in touch.
Photo by Jonathan Simcoe via Unsplash, used with permission.
