Questionnaires are appropriate as a method for data collection when resources are limited. They are useful for eliciting information from a large group of people, if this target sample is accessible, willing, and able to complete the questionnaire. This approach works best when the questions are clearly defined, easily understood, and directly relevant to the research objectives.

There are two key elements to successful questionnaire design: content and structure. Content relates to the nature and type of questions used to gather the data that you need, while structure is about how questions are organised. This is to ensure clarity, logical flow, and ease of response for participants. A well-structured questionnaire minimises the cognitive burden of the respondent. It maintains their engagement. In addition, it improves the accuracy and reliability of the data that has been elicited by means of the questionnaire.

This is very basic knowledge. It should be the grounding of anybody who is engaged in the iterative design of a questionnaire, a survey or a clinical assessment tool. To illustrate how remedial this is, I have selected advice about questionnaire design from five separate undergraduate, 101-level research methods books and they state the following:

·       Use precise, concise and simple language

·       Do not use jargon, abbreviations or acronyms

·       Do not include questions that present any degree of ambiguity

·       Do not use double-barrelled questions

·       Do not design questions that are negative constructions

·       Be mindful of the cognitive burden on the respondents: do not have long questions loaded with multiple concepts

·       For the same reason, do not stretch respondents’ memories.

Steven Pinker, in Words and Rules (1999) makes an important point about syntax in questionnaires. The structure of questionnaires needs to be simple. People living with a cognitive impairment may have difficulties with complex syntax, even when the vocabulary is easy to understand. For example, Pinker states that people with Parkinson’s disease might find it difficult to decode utterances such as “it was the boy that the girl tickled” and “the eagle that the hawk chased was fast.” Therefore, developers of instruments need to take advice such as this into account.

Over the years, we have tested a wide range of Clinical Outcome Assessments: some good, some bad, some very ugly. Below are eight tips for best practice advice in questionnaire design applied to Clinical Outcome Assessments:

1.       A clear understanding of the objectives of the assessment: Objectives for the questionnaire need to be defined and clearly understood from the outset to make sure that it addresses specific clinical outcomes.

2.       Knowing the target audience: Items need to be tailored to the target audience’s literacy, cultural background, and cognitive abilities. However, this is where there is a tension between standard practice, i.e. one-size-fits-all and the real needs of the target audience. Failing to do this, such as by employing complex medical jargon or culturally insensitive language in the items may alienate the user.

3.       Design items that are clear. Developers should use clear, concise and unambiguous language in their assessments. This not only applies to the question items, but also the response choices, the instructions and, in eCOAs, functions, such as ‘save’, ‘next’ and so on. How instructions and functions are understood is often overlooked in exercises such as cognitive debriefing.

4.       Response options need to be consistent and appropriate to the study and target population. Has the response scale that you are intending to use, such as a Likert scale or Visual Analogue Scale, been validated for the target clinical population?

5.       Make sure the items are relevant. Items need to be sensitive to the specific clinical context where it is intended to be used. The questions need to be relevant enough to detect meaningful changes in the tool. Irrelevant questions or ones that are too vague may not detect changes in the patients’ clinical status.

6.       Think about length and cognitive burden. The questionnaire needs to be as short as possible while still capturing the data that is required to minimise any possibility of participant fatigue. This is not only to do with the number of items, but also the length and complexity of each question. Otherwise, if patients are overloaded with lengthy or repetitive questions, this may lead to answers that are incomplete or inaccurate. The same applies to how much information (or concepts) are introduced into each item.

7.       Consider the emotional impact on participants. Questions need to be respectful, non-invasive and ethical. This is especially so for sensitive health topics where an intrusive question may have a negative emotional impact on the participants.

8.       Work out in advance what the best mode of delivery will be. The mode of administration, such as paper, digital, self- or clinician-administered, should be selected based on what is most appropriate for the target audience. This needs to be considered before the development of the instrument. Do not make assumptions about mode of delivery: eCOAs may not be appropriate for everyone. There is a digital divide and, very often, vulnerable patient groups are on the wrong side of it. Choose an administration mode that is convenient and accessible for the target population.

Following these eight best practice tips can lead to Clinical Outcome Assessments that are useful.

The next article will focus of instances of poor practice in COA design.

Thank you for reading!

Mark Gibson

Leeds, United Kingdom, February 2025