By Damian Zboriowski (Predictive Solutions)

“The data collection system must support the researcher at every stage of the project.” However, before systemic support can begin, it is imperative to complete the planning phase, which includes stages such as: preparation of the research plan, selection of techniques and trials, conceptualization, operationalization, etc.

I strongly advise against immediately sitting at the computer and trying to materialize ideas you have in your head into a questionnaire. Such an approach is always subject to multiple modifications with double the work necessary. The moment you launch the questionnaire design software program should be the moment when you have the final content of the questions “on paper” along with the full logic, i.e. transition rules, logical conditions, established order of questions or blocks, etc. Of course, when piloting the questionnaire there are likely to be several changes and corrections required, however, their number will be much smaller.

I believe that a good research project must be carefully executed. Lack of care with naming conventions of questions, chaotic descriptions and comments, and poorly thought-through questionnaire logic, will not prevent the survey from happening, but it will certainly make it harder to work with in the future. A key aspect when building a survey is that the system is creating the structure of the data in the background (and saving it to a database). Needless to say, a carelessly prepared questionnaire will lead to a poorly structured data set, which will have a knock-on effect on the final survey results. It is therefore essential that structural elements such as variable names, variable and value labels, missing data, variable format, etc, are determined at the beginning of the survey design process, not later on when the survey is in full swing. (The topic of the result data set and our ability to influence it at the design stage is an interesting yet complex one. For this reason, I will devote a separate article to it in the future).

When it comes to building the digital questionnaire, I will be using PS QUAESTIO PRO which is an on-premise system that allows you to construct surveys and publish them in real time, e.g. on a web server or mobile device. The program is equipped with numerous design wizards that allow you to create virtually any type of question with logic transitions and fills. However, my goal here is not to describe every feature/function of the program as these are well documented in the user manual. In this article, I will present a few key elements that you may not see at first glance, but will surely encounter sooner or later in your work.

First of all, the greatest art is the skilful transfer of the content of the questions from the “paper version” to the survey system. By “paper version” I am referring to the version prepared in MS Word, OpenOffice or any other text editor where the content has been prepared with no research principles in mind, and with no interactions with other questions. When it comes to the actual design of the questionnaire, I strive for maximum simplicity and ease of completion. Remember that for the researcher, after the hundredth reading of the content of a given question, everything starts to be easy and clear, while the respondent will see the questionnaire only once. If we give the respondent even the smallest reason to doubt when answering, we are already losing the battle for high quality results. A good practice is to show the questionnaire to someone who sees it for the first time and who is not familiar with the research topic. Some of the comments and observations you receive back may surprise you and give you a completely different perspective on the content of the questionnaire.

CHECKBOXES

When it comes to ease of completion, there are many ways a question can be modified during the authoring stage. See the example below:

Unlike the first question, the second question requires the respondent to select only those activities he undertook thereby easing completion and saving him valuable time.

It is not my goal here to teach you how to construct questions, or to break down individual questions and analyze their key components. My goal is to draw your attention to the idea of simplicity, as it is very often marginalized or completely overlooked by the researcher. The idea is that the easier the questionnaire is perceived to be, the greater the chance the respondent will not only complete the questionnaire, but will avoid further questionnaires down the road.

OMITTING SPECIFIC QUESTIONS

Here is another example where the simplification process takes on a different form.

Sample question:

In this case, we are dealing with two questions simultaneously. Firstly, the respondent indicates all the reasons why he started a given course, and secondly, he has to choose the most important one. There is potential here for the respondent to make mistakes. First, what if he confuses the columns and answers back to front? Secondly, he may indicate only one reason for starting a degree program in which case the additional question about the most important is not needed. (I do not discuss here so-called incorrect completion, i.e. intentional or accidental non-compliance with the instructions contained in the questionnaire. I will cover this topic in more detail in another article, because the system offers a lot of additional control possibilities in this area). There are several ways to simplify the above question. My approach divides the question into two separate questions. The first will only ask about the reasons for choosing a given course:

The second one asks for the most important reason, but only in relation to the answers given in the previous question.

What if the respondent selects only one answer in the first question? If the second question is not displayed the respondent does not waste time on it, however (more important for the researcher), an appropriate answer to the second question will not appear in the result data set. However, if the respondent does not indicate the most important reason, it is implicitly the answer from the preceding question. To achieve this in the questionnaire, we start by adding a special IFBlock object in the Condition section:

Objects of this type allow for the conditional display of sets of questions. In our case, the condition for which the most important reason will not be displayed is only one answer was given in the previous question about the reasons for choosing a given field of study. For simplicity, we use the question codes as follows:

p2 – What made you choose this particular field of study?
p3 – Please indicate the most important reason

It is enough at this point to select question p2 with the condition includes at least and enter the value 2, as seen below:

From now on, the question p3 will not be displayed to any respondent who selected only one answer in the question p2. The last thing to deal with is the automatic insertion of the answer to question p3 to all respondents for whom the above condition did not apply. To do this, we add an additional Else object that will run whenever the basic IFBlock condition is not met. The Else object serves as a complement to the base condition allowing for a range of outcomes, e.g., the appearance of an additional alternative question, or a non-standard message. In our case, we will use the Set Response object:

In doing so, the researcher is now assured of responses to any questions that are omitted (not displayed to the respondent). Such objects are most often used for questions omitted by transition rules, e.g., by automatically selecting the category not applicable as an answer to a question. Since in our case we want the answer selected in question p2 to be automatically copied to question p3, we modify the previously inserted Set Response object as shown below:

As you probably noticed, you only need to put the name of the p2 question in the Build a response field. Now it is enough to perform a few test completions to make sure that that part of the survey works correctly. The graphic scheme should look like this:

In taking these steps, we have eliminated all the problems I described earlier, and at the same time significantly increased the transparency of the questionnaire.

GROUPING QUESTIONS ON A SCALE

Sometimes simple tricks that use individual program functions in a non-standard way allow you to get very interesting final effects. For example:

Here, grouping questions and creating Sublists allows you to obtain the effect of the so-called “Semantic differential”, i.e., questions on a scale where respondents must evaluate a given phenomenon, indicating the intensity of two opposite terms. By default, the program allows you to put answers in rows or columns, but the description of each category is on the right side of the checkbox, i.e.

We therefore need to slightly modify our question. It is enough to mark all the answers and group them using the Group into Sublist function, while setting the Orientation option to Rows in the Presentation section.

If we want the number of questions based on the differential scale to be greater, we can additionally group them on one page.

DISPLAYING EARLIER ANSWERS

Another simple, yet interesting example may be the need to display the answers that the respondent gave in the two previous questions.

For example, we ask the respondent about her favorite coffee, presenting the list of main brands available on the market. Then, we ask her to indicate the brands she has heard of. Both questions have an identical range of answers. Finally, we ask a question which contains only those coffee brands that have previously been selected, asking to select only those that she has ever bought. Displaying only those answers from the previous question is relatively simple and is based on the standard function Filter: Chosen located in the Responses section.

In order to be able to display the answers from more than one question, it is necessary to additionally enter the notation as in the figure below:

In order to be able to display the answers from more than one question, it is necessary to additionally enter the notation as in the figure below:

Where p1 and p2 are the names of the questions.

ERROR MESSAGES

The last example that I will present is a method that safeguards against the respondent’s “laziness”. Let’s assume that we plan to ask a question about the name of the region from which the respondent comes. For example:

Since the list of categories is quite long, I suggest using a dropdown box instead of classic checkboxes. Unfortunately, in this case, the first of the available categories will become the default and will be automatically displayed in the selection window.

The risk here is that an inattentive respondent will not modify it and will leave the default value, which unfortunately will result in entering an incorrect answer. One solution is to insert an additional value at the beginning of the category list and set the status to Missing.

In so doing we achieve two things: we draw the attention of the respondent to the response list more, and, even if the category is included in the result set, it will be marked as missing data. However, it is a solution that does not completely fulfil its task. Therefore, I propose a simple trick. It is enough to use the Group into Sublist option once again, selecting all available categories beforehand. Then we modify the default list name to

As a result, the default value displayed in the prompt is the list name. Since this is not an answer, we receive an appropriate message when we try to move to the next question:

SUMMARY

As you can see from the examples above, there are many interesting options for non-standard use of ready-made program functions. This is important especially where the researcher does not have programming skills and relies on a graphical environment such as the one offered by PS QUAESTIO PRO.

There may be situations beyond our control, when we will be forced to reproduce the layout of the “paper” version (e.g. the client requires us to do so), and we do not know how to achieve such an effect. Each program based on a graphical interface has a finite number of options and sooner or later a need will arise that cannot be “clicked”. Fortunately, PS QUAESTIO PRO features a Professional development environment where complex survey design can be achieved using the scripting language. In addition, it is possible to prepare management scripts, e.g. downloading data about respondents and transferring them to a research project. We can also build automatic reporting machines for many recipients at the same time. An additional advantage of this solution is the ability to prepare ready-made scripts / wizards that the researcher can use in the Author program (used in these examples).

I hope this handful of tips will allow you to take a different look at some areas of your work and rethink how you operate. If you manage to improve your tasks a little and improve the final result, I will assume that my goal has been achieved.

2 Comments

google on June 30, 2021 at 10:16 pm
Hello, this weekend is pleasant designed for me, as this point in time i am reading
this impressive informative paragraph here at my home.
google on July 5, 2021 at 11:10 am
Hello there, You’ve done an incredible job. I’ll certainly digg it and personally recommend to my friends.
I’m sure they’ll be benefited from this web site.

DESIGNING A QUESTIONNAIRE: HOW TO CONVERT THE CONTENT OF A SURVEY INTO DIGITAL FORM