TPRM Blog 7- The Do's and Don'ts For Making Questions Answerable
Author: GRC Practice Director, Alex Hollis.
In this Third Party Risk Management blog series, Alex Hollis will guide you through developing effective information gathering for third parties using five key steps to formulate a third-party questionnaire. The webinar is available on-demand via BrightTALK here, or take a look at our full Third-Party Risk Management services.
There are five key steps to the formulation of a third-party questionnaire:
- Requirements – establishing the needs of the organisation both in terms of the risks that need to be managed, the compliance needs from regulation, and any stakeholder commitments
- Research – obtaining an understanding of the types of information needed to satisfy the requirements and prioritising the needs among the various types of third parties the organisation has
- Planning – consideration for the method, structure, and number of assessments (this can also include non-questionnaire approaches such as audits and interviews)
- Writing questions – Formulating the actual questions themselves and the method of response
- Testing – Obtaining validation and identifying any areas of improvement
In this seventh instalment, Alex will explore the do’s and don’ts of writing answerable questions. This will include the importance of allowing respondents to communicate their uncertainty and uses real-life questionnaire examples.
1) Do – State timeframes
| Has there been an information security breach? | Yes/No |
Here the timeframe has been left open to interpretation. The designer intends to ask if there has ever been a security breach; however, a respondent could interpret it as a breach within the last year or as far as they can remember.
2) Don’t – Assume regularity of behaviour
When posing questions, don’t force a regular schedule on something which may be conducted sporadically.
| How frequently have DR/BC tests been performed? | less than annually, 1-2 annually, 3+ annually. |
A company that doesn’t do regular annual testing but instead does three full tests in a single year every 2-3 years might take an average, selecting ‘1-2 annually,’ or they might answer ‘less than annually’.
Removing this interpretation step by asking prevents any accidental or deliberate misinterpretation. Instead, ask: How many tests have been performed in the last three years? Or: When was the last test conducted?
3) Don’t – Ask people for information they don’t have
There is often an assumption that the customer can ask any question, and the third-party will have all the answers. The more specific the question, the more effort there will be to obtain the requested data.
| What is the average page loading time customers have experienced in the past 7 days? |
This will likely result in the third-party not answering the question itself but instead redirecting to another area they do have data for or making a less accurate estimation of the answer. Neither of which are particularly useful.
4) Do – Make sure that questions are appropriate for the respondents
| Do you follow a secure software development lifecycle? |
This question assumes that the third-party you are questioning provides you with a developed software product. You may be using some third parties for services only, or for services that are backed by products they do not themselves develop. Forcing an answer to this question is not appropriate.
5) Do – Allow respondents to say “Don’t know”, “Uncertain” or “N/A.”
Following on from the above questions being appropriate, in some circumstances, you might be asking questions of respondents that you cannot determine in advance whether they will be appropriate. If you cannot avoid asking the questions, allow the respondent to express that they do not understand or that the question is inappropriate.
When you come across instances like this, evaluate whether others would benefit from an update to the structure of the question or if it would better target different groups of third-parties.
In TPRM Blog 8…
Alex will be exploring Krosnick’s research on respondent’s experience to answering questions to provide a guide on how to increase your respondents’ level of engagement.
View the previous blogs in our Third-Party Risk Management series, or head over to our Vendor Risk Management capability to see how SureCloud could help you bolster the security of your partnerships.
