TPRM Blog 7- The Do's and Don'ts For Making Questions Answerable

Author: GRC Practice Director, Alex Hollis.

Blog Series Introduction

In this Third Party Risk Management blog series, Alex Hollis will guide you through developing effective information gathering for third parties using five key steps to the formulation of a third party questionnaire. The webinar is available on-demand via BrightTALK here.

There are five key steps to the formulation of a third party questionnaire:

• Requirements – establishing the needs of the organisation both in terms of the risks that need to be managed and the compliance needs from regulation and any stakeholder commitments.
• Research – obtaining an understanding of the types of information needed to satisfy the requirements and prioritising the needs among the various types of third parties the organisation has.
• Planning – consideration for the method, structure, and number of assessments (this can also include non-questionnaire approaches such as audits and interviews)
• Writing questions – Formulating the actual questions themselves and the method of response.
• Testing – Obtaining validation and identifying any areas of improvement.

In the seventh instalment, Alex will explore the do’s and don’ts on how to write answerable questions. This will include the importance of allowing respondents to communicate their uncertainty. The blog includes real life examples.

1) State timeframes

Here the timeframe has been left open to interpretation. The intent from the designer is asking if there has ever been a security breach; however a respondent could interpret that as being a breach within the last year, or as far as they can remember.

2) Don’t assume regularity of behaviour

When posing questions don’t force a regular schedule on something which may be conducted sporadically.

A company who doesn’t do regular annual testing but instead does three full tests in a single year every 2-3 years might take an average, selecting ‘1-2 annually’ or they might answer ‘less than annually’.

Removing this interpretation step by asking how many tests have been performed in the last three years? or when was the last test was conducted? removes the potential for the question to be accidentally or deliberately misinterpreted.

3) Don’t ask people for information they don’t have

There is often an assumption made that customer can ask any question, and the third party will have all of the answers. The more specific the question, the more effort there will be to obtain the requested data.

This will likely result in the third party not answering the question itself but instead redirecting to another area they do have data for or in making a less accurate estimation of the answer. Neither of which are particularly useful.

4) Make sure that questions are appropriate for the respondents

This question makes the assumption that the third party you are questioning provides you with a software product which is developed. You may be using some third parties for services only, or for services which are backed by products they do not themselves develop. Forcing an answer to this question is not appropriate.

5) Allow for respondents to say “Don’t know”, “Uncertain” or “N/A.”

Following on from the above questions being appropriate, in some circumstances you might be asking questions of respondents that you and you cannot determine in advance whether they will be appropriate. If you cannot avoid asking the questions, allow for the respondent to express that they do not understand or that the question is not appropriate.

When you come across this evaluate whether others would benefit from an update to the structure of the question or the targeting of that question towards certain groups of third parties.

Next Week…

Stay tuned for the next blog in this series “Make Questions Easy”. Alex will be exploring Krosnick’s research on respondent’s experience to answering questions.

To view the previous blogs in the series click here.
See you next week!