How individuals interpret and respond to survey questions is influenced by cultural norms, social expectations and cognitive processing styles. These differ from culture to culture. Therefore, when translating and adapting Clinical Outcome Assessments (COAs) intended for culturally and linguistically diverse populations, there needs to be careful consideration of the response biases and cognitive differences. These arise because of underlying cultural values. If not properly addressed, these differences can introduce systematic bias. In turn, this affects how accurate the responses are and how well they can be compared across study sites.

This article looks at five key challenges that focus on response biases. These are:

-            Response format and scale usage

-            Social desirability bias

-            Self-reporting and modesty bias

-            The influence of authority

-            Cognitive processing differences across cultures

Response format and scale usage

People from different cultures have varying preferences for the kind of response formats they are most used to and most comfortable with. For example, a lot of COAs involve Likert Scales, with response such as ‘Strongly Agree to Strongly Disagree). This is because Western cultures are used to them and prefer them. It follows, then, that Likert Scales are commonly found in COAs because they tend to originate in Western societies. However, it should not be assumed that people in other cultures are familiar with them.

For example:

-            In some Asian and Middle Eastern cultures, the preference might be for ‘Yes/No’ responses, rather than graded scales. This would naturally mean the entire structure of the item would have to change in order to accommodate a binary response.

-            In Latin American and Southern European cultures, some people may be more inclined to select extreme scale points

-            By contrast, in Scandinavian cultures, participants in a study may tend towards midpoint responses and prefer to avoid extremes.

Imagine two participants in a clinical study, one in Sweden and the other in Peru. Imagine that they are both experiencing very similar intensities of pain, yet they respond differently on a 10-point scale: the participant in Sweden responding with a ‘6’ or a ‘7’, while their Peruvian counterpart rating it an unquestionable ‘10’. How do you account for such a subjective discrepancy? 

In COA development, at source, more consideration needs to be made about how people around the world differ in their response preferences and this would impact on how the question is constructed. There cannot be a simple assumption that what works in the source culture will apply everywhere. It does not.

Social Desirability Bias

Some topics, such as mental health, substance use, socioeconomic background, may carry stigma in some cultures. Mindful of this when completing assessments, participants in a clinical study may underreport symptoms or experiences to conform to societal norms and to avoid negative judgements.

For example, in some Middle Eastern and Asian cultures, there is a high stigma around mental health issues. This can lead to study participants underreporting items to do with depression or anxiety. In collectivist societies, the stigma is also transferred to those associated with the person - family, friends. This means that any admission to personal struggles that carry the baggage of stigma may affect their reputation as well. In clinical assessments, this might be a motivation for the study participant to provide more socially acceptable responses.

During the COA development phase, one way to address this is to incorporate indirect questioning techniques and frame them in a way that they are neutral and non-judgmental, softening the directness to the study participant. For example. “Some people feel anxious in social situations like going out to meet new people. How often does this happen to you?” Another strategy could be to emphasise anonymity and confidentiality in data collection to reduce societal pressure.

Self-reporting and Modesty Bias

In collectivist cultures, individuals may underreport difficulties so that they do not appear weak or dependent. This is called modesty bias and can lead to underestimations of health conditions or limitations. For instance, in Japanese culture, people may downplay pain or discomfort or not disclose it at all so that they do not burden other people. Alternatively, some African cultures that put a premium on resilience and endurance may lead to individuals underreporting their levels of pain or distress.

A solution could be to construct questions in a way that normalises difficulties, not making the questions directed at them: “Many people feel pain when they do daily activities. How often do you experience this?” Alternatively, questions could be framed in the third person to reduce the potential for self-consciousness, e.g. “In your community, how common is this issue?” However, would this question fly in Western contexts? The more I consider cross-cultural applicability of COA item design, the more a ‘one size fits all approach’ seems very difficult to achieve.

Influence of Authority and Social Hierarchy

Some cultures have strong, if not rigid, hierarchical social structures. The perceived power differential between study participants and interviewer might cause the participant to alter responses. This is to do with face maintenance, which we have covered in other articles. In a clinical assessment context, participants may feel pressured to provide socially acceptable responses, aligning with expectations rather than honest ones. Similarly, in Middle Eastern cultures, a participant may not want to disclose mental or physical health struggles if they are interviewed by someone perceived as an authority figure. Or in Latin America, where participants may answer in a way that they think the interviewer wants to hear, out of deference to the medical profession.

A solution to this could be self-administered COAs, rather than face-to-face interviews. In addition, interviewers need to be professionally trained to use a neutral, non-authoritative tone and to reassure the participants that honest answers are valued.

Differences in Cognitive Processing Styles

Developers of COAs must never assume that people around the world follow the same cognitive processes when reading through a clinical questionnaire or eDiary. Culture, script and how information is presented all influence how people interpret survey questions and organise information. Some cultures emphasise analytical thinking, such as how details are broken down, while other cultures rely on holistic thinking, e.g. considering the broader context.

For example, Western cultures tend to process information analytically and focus on specific elements of a question, one question at a time. By contrast, East Asian cultures process texts holistically, considering context and relationships, rather than isolated details. Meanwhile, Indigenous and African cultures may emphasise responses that are based on narrative and experience, rather than structured scales.

COA developers need to structure questions in a way that accommodates both analytical and holistic thinkers, by using straightforward phrasing that minimises the cognitive load. Examples and context could be provided to help study participants frame their answers appropriately. As part of the development process (and not wait until the Linguistic Validation stage), there could be cross-cultural testing of COA drafts to refine iteratively question formats that work across diverse cognitive styles.

Conclusion

Response biases and cognitive differences play a huge role in how individuals engage with COAs. When COAs are not properly adapted, these factors can cause systematic errors that reduce the reliability and validity of the data.

By considering the points raised in this article, COA developers can design assessments that capture more accurate and culturally relevant data.

Thank you for reading,

Mark Gibson

Leeds, United Kingdom, March 2025