By Damian Zboriowski (Predictive Solutions)
“The data collection system must support the researcher at every stage of the project.” However, before systemic support can begin, it is imperative to complete the planning phase, which includes stages such as: preparation of the research plan, selection of techniques and trials, conceptualization, operationalization, etc.
I strongly advise against immediately sitting at the computer and trying to materialize ideas you have in your head into a questionnaire. Such an approach is always subject to multiple modifications with double the work necessary. The moment you launch the questionnaire design software program should be the moment when you have the final content of the questions “on paper” along with the full logic, i.e. transition rules, logical conditions, established order of questions or blocks, etc. Of course, when piloting the questionnaire there are likely to be several changes and corrections required, however, their number will be much smaller.
I believe that a good research project must be carefully executed. Lack of care with naming conventions of questions, chaotic descriptions and comments, and poorly thought-through questionnaire logic, will not prevent the survey from happening, but it will certainly make it harder to work with in the future. A key aspect when building a survey is that the system is creating the structure of the data in the background (and saving it to a database). Needless to say, a carelessly prepared questionnaire will lead to a poorly structured data set, which will have a knock-on effect on the final survey results. It is therefore essential that structural elements such as variable names, variable and value labels, missing data, variable format, etc, are determined at the beginning of the survey design process, not later on when the survey is in full swing. (The topic of the result data set and our ability to influence it at the design stage is an interesting yet complex one. For this reason, I will devote a separate article to it in the future).
When it comes to building the digital questionnaire, I will be using PS QUAESTIO PRO which is an on-premise system that allows you to construct surveys and publish them in real time, e.g. on a web server or mobile device. The program is equipped with numerous design wizards that allow you to create virtually any type of question with logic transitions and fills. However, my goal here is not to describe every feature/function of the program as these are well documented in the user manual. In this article, I will present a few key elements that you may not see at first glance, but will surely encounter sooner or later in your work.
First of all, the greatest art is the skilful transfer of the content of the questions from the “paper version” to the survey system. By “paper version” I am referring to the version prepared in MS Word, OpenOffice or any other text editor where the content has been prepared with no research principles in mind, and with no interactions with other questions. When it comes to the actual design of the questionnaire, I strive for maximum simplicity and ease of completion. Remember that for the researcher, after the hundredth reading of the content of a given question, everything starts to be easy and clear, while the respondent will see the questionnaire only once. If we give the respondent even the smallest reason to doubt when answering, we are already losing the battle for high quality results. A good practice is to show the questionnaire to someone who sees it for the first time and who is not familiar with the research topic. Some of the comments and observations you receive back may surprise you and give you a completely different perspective on the content of the questionnaire.
When it comes to ease of completion, there are many ways a question can be modified during the authoring stage. See the example below:
OMITTING SPECIFIC QUESTIONS
Here is another example where the simplification process takes on a different form.
Objects of this type allow for the conditional display of sets of questions. In our case, the condition for which the most important reason will not be displayed is only one answer was given in the previous question about the reasons for choosing a given field of study. For simplicity, we use the question codes as follows:
- p2 – What made you choose this particular field of study?
- p3 – Please indicate the most important reason
It is enough at this point to select question p2 with the condition includes at least and enter the value 2, as seen below:
GROUPING QUESTIONS ON A SCALE
Sometimes simple tricks that use individual program functions in a non-standard way allow you to get very interesting final effects. For example:
DISPLAYING EARLIER ANSWERS
Another simple, yet interesting example may be the need to display the answers that the respondent gave in the two previous questions.
For example, we ask the respondent about her favorite coffee, presenting the list of main brands available on the market. Then, we ask her to indicate the brands she has heard of. Both questions have an identical range of answers. Finally, we ask a question which contains only those coffee brands that have previously been selected, asking to select only those that she has ever bought. Displaying only those answers from the previous question is relatively simple and is based on the standard function Filter: Chosen located in the Responses section.
In order to be able to display the answers from more than one question, it is necessary to additionally enter the notation as in the figure below:
The last example that I will present is a method that safeguards against the respondent’s “laziness”. Let’s assume that we plan to ask a question about the name of the region from which the respondent comes. For example:
The risk here is that an inattentive respondent will not modify it and will leave the default value, which unfortunately will result in entering an incorrect answer. One solution is to insert an additional value at the beginning of the category list and set the status to Missing.
In so doing we achieve two things: we draw the attention of the respondent to the response list more, and, even if the category is included in the result set, it will be marked as missing data. However, it is a solution that does not completely fulfil its task. Therefore, I propose a simple trick. It is enough to use the Group into Sublist option once again, selecting all available categories beforehand. Then we modify the default list name to
As a result, the default value displayed in the prompt is the list name. Since this is not an answer, we receive an appropriate message when we try to move to the next question:
As you can see from the examples above, there are many interesting options for non-standard use of ready-made program functions. This is important especially where the researcher does not have programming skills and relies on a graphical environment such as the one offered by PS QUAESTIO PRO.
There may be situations beyond our control, when we will be forced to reproduce the layout of the “paper” version (e.g. the client requires us to do so), and we do not know how to achieve such an effect. Each program based on a graphical interface has a finite number of options and sooner or later a need will arise that cannot be “clicked”. Fortunately, PS QUAESTIO PRO features a Professional development environment where complex survey design can be achieved using the scripting language. In addition, it is possible to prepare management scripts, e.g. downloading data about respondents and transferring them to a research project. We can also build automatic reporting machines for many recipients at the same time. An additional advantage of this solution is the ability to prepare ready-made scripts / wizards that the researcher can use in the Author program (used in these examples).
I hope this handful of tips will allow you to take a different look at some areas of your work and rethink how you operate. If you manage to improve your tasks a little and improve the final result, I will assume that my goal has been achieved.