We often see questionnaires that contain exploratory open-ended questions. This is no surprise, because when people are studying a topic, they want to know as much as possible about it. Below are examples of exploratory open-ended questions:
“What did you like most about . . . ?”
“What did you like least about . . . ?”
“What would you like to see changed?”
“How did you make that decision?”
“If you said YES to the previous question, please explain why you said YES.”
“If you said NO to the previous question, please explain why you said NO.”
These questions are certainly important. The problem is that these are qualitative questions.
In part 1 of my book, “Plan Research,” I talked about the importance of addressing qualitative issues with qualitative methods. I also explained why you need to conduct qualitative research prior to writing a questionnaire, as many of the issues you need to explore can be addressed only with qualitative methods.
If you have questions like these, you will get much better information and insight by conducting qualitative research, where interviewers can interpret initial responses, follow up with probing questions, ask respondents for clarification, pose scenarios to help uncover deeper levels of thinking, and so on.
In general, avoid using exploratory open-ended questions as a substitute for qualitative research. As an aside, one of the key issues in survey research is to keep questionnaires short. Putting too many exploratory open-ended questions in a survey makes the questionnaire long and difficult for respondents, which leads to higher dropout rates. This is especially true with mobile administration of surveys.
Please understand that I am not saying never add exploratory open-ended questions to surveys. We all have situations that inspire or require it. What I am saying is do not use exploratory open-ended questions as a substitute for qualitative research.
And have realistic expectations about what information you will get and interpret the results accordingly.