Clinical Outcome Assessments are developed at great expense. If I could ‘guestimate’, in US dollars, the cost of each COA development phase can be as follows:

·       Initial planning and design: 50,000 to 150,000 USD. This would include stakeholder meetings and workshops, literature review and gap analysis, questionnaire design and item development and expert consultation.

·       Qualitative research and qualitative interviewing: 80,000 to 200, 000 USD. The activities comprised at this stage would be focus groups, interviews, cognitive debriefing (e.g. concept elicitation), analysis and reporting.

·       Pilot testing and validation: between 100, 000 to >500,000 USD. This includes small-scale pilot testing, psychometric validation studies and cross-cultural adaptation.

·       Activities around regulatory submissions: 50,000 to 150,000 USD. This typically includes preparation of documentation for submission, such as to FDA or EMA, and before that, there would be some kind of regulatory consultation and strategy.

·       Digital platform development: 100,000 to more than 500,000 USD. These activities would subsume the design of the User Interface, software development, HIPAA/GDPR data security and compliance, UX testing and quality assurance.

·       Miscellaneous costs: around 200,000 USD. This would include translation and cultural adaptation, legal and IP considerations, project management, publication and dissemination.

There are a number of factors that influence the development costs. These include the type of COA, i.e. whether PRO, ObsRO, PerfO measures, and so on. It can also depend on the complexity of the disease or condition it is designed for, the number of languages for adaptation, sample size and geographic scope, as well as regulatory requirements and the complexity of the digital platform being developed. Regarding geographic scope, this will only increase, as more developers are wising up to the fact that there needs to be more diversity of sample cohorts. Therefore, the overall costs range from half a million to more than two million US dollars. This is for an individual Clinical Outcome Assessment.

With this level of investment, why is it, then, that there is so much variation in the quality of Clinical Outcome Assessments? This next batch of articles are on the theme of current practices in the design of COAs. Over the years we have tested thousands of them, and we have a feel for good and poor practice. One of the areas that is most deficient in the development of COAs, is, inexplicably, the actual wording of the questions. A large part of this is a lack of awareness of the principles of Information Design and questionnaire is just one part of this. It is vital that developers start incorporating Information Design expertise in their COA and eCOA development, from the outset. A questionnaire that displays problems in the source language will inevitably carry these issues over in translation. In fact, the translation process may even magnify problems of question structure and wording. Often, these problems are not discovered or identified using cognitive debriefing as a method of testing patients’ understanding of each item.

Given this substantial investment in COA development, clarity and precision in question design should not be an afterthought. It is essential. Without it, there is a risk that the very data that these assessments are built to capture are undermined.

The next article will examine basic survey design, with a focus on question construction.

Thank you for reading,

Mark Gibson

Leeds, United Kingdom, February 2025