Auditing customer feedback in quality systems
The customer feedback process is a critical part of the quality management system, and should therefore receive adequate attention during a third-party audit. Feedback from the customer is one of the primary performance indicators that can be used to judge the overall effectiveness of the quality management system (QMS). It is important therefore that the auditor verify that:
- The organisation’s customer communication channels promote an adequate awareness of the process by which customers can provide feedback;
- Inputs to the customer feedback process include relevant, representative and reliable data;
- This data is analysed effectively; and
- The output from this process provides useful information to the management review and other QMS processes, to enhance customer satisfaction and drive continual improvement.
The overall objective of ISO 9001: 2000, as stated in clause 1.1, is to specify requirements for a quality management system where an organisation:
- Needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements; and
- Aims to enhance customer satisfaction through the effective application of the system, including processes for continual improvement of the system and the assurance of conformity to customer and applicable regulatory requirements.
Clause 7.2.3 requires the organisation to “determine and implement effective arrangements for communicating with customers in relation to ... customer feedback, including customer complaints.”
Clause 8.2.1 of ISO 9001:2000 states: “ As one of the measurements of the performance of the quality management system, the organisation shall monitor information relating to customer perception as to whether the organisation has met customer requirements. The methods for obtaining and using this information shall be determined.”
The ISO/TC176 guidance document on terminology (ISO/ TC176/SC2/N526R) emphasises that monitor means “to observe, supervise, keep under review; measure or test at intervals”.
It is important for auditors to recognise there is no specific requirement in ISO 9001:2000 clause 8.2.1 for the organisation to perform formal customer satisfaction surveys, or other measurements of customer satisfaction, although this could of course be a useful tool in monitoring customer perceptions. It is therefore important that the organisation tries to see things from the customer’s perspective, and monitors the customer’s perceptions; measurement of customer satisfaction may be appropriate in some situations, but it is not a direct requirement of the standard.
In addition to these specific references to the customer feedback process, there are a number of indirect references throughout the whole of the standard, which the auditor needs to take into account. Examples include feedback as part of the design and development process, process validation and others.
What should be addressed when auditing customer feedback processes?
Customer feedback is a process. It needs to be audited as a process, not as a “clause of the standard”. An evaluation also needs to be performed on the way in which the process is managed (see ISO 9001:2000 clause 4.1.c), and its ability to provide meaningful information with which to judge the overall effectiveness of the QMS. The way in which the organisation obtains this feedback (“the method”) is up to its management to define. The auditor should therefore be aware of the many factors that can affect the organisation’s approach, and should recognise there is no fixed recipe.
Due consideration should be given to factors such as:
- Organisation size and complexity;
- Degree of sophistication of products and customers;
- Risks associated with the product; and
- Diversity of customer base.
Before the audit of the customer feedback process
The auditor needs to be aware of the specific characteristics of the organisation’s products that are likely to impact customer satisfaction. Throughout the audit the auditor should be alert for indications of customer satisfaction or dissatisfaction that could serve as inputs to the audit of the customer feedback process. Good sources of such information may include:
- Goods returned by the customer
- Warranty claims
- Revised invoices
- Credit notes
- Articles in the media
- Consumer websites C
- Communication with the customer.
During the process assessment
These are some of the issues an auditor should address during an audit of the customer feedback process:
- What is the desired output of this process?
- What information is actually available on customer perceptions?
- How is this information used by management to drive improvements to the product, processes and the QMS?
- Are all customer categories covered by this information?
It is important to remember that the organisation may have more than one category of customer - see the definition of “customer” in ISO 9000:2000 clause 3.3.5. For example, a manufacturer may sell to wholesalers, who then sell to retailers, who in turn sell to the general public. In this case the organisation may need to address all three types of customer and they may all have different perceptions. The organisation could be satisfying one group and upsetting another.
How is the data collected to feed the process?
There are many ways for an organisation to monitor its customers’ perceptions, and the auditor should avoid preconceived ideas about how this should be done. Techniques the organisation can use include:
- Face-to-face evaluations, which may be appropriate in many service organisations, such as hotels – “ How was your stay with us? ” or restaurants – “I hope you enjoyed your dinner ”;
- Telephone calls or periodic visits to customers;
- Questionnaires or surveys carried out by the organisation itself, or by independent market researchers;
- Other contact with customers, for example by service or installation personnel;
- Internal enquiries among the organisation’s personnel who are in contact with customers;
- Evaluation of repeat business;
- Monitoring accounts receivable, warranty claims; and
- Customer complaints analysis.
Often complaints are the only spontaneous feedback received from customers, and these should be analysed for any trends, key concerns and impacts. It must be stressed, however, that customer complaints cannot be the only input for monitoring customer perceptions. Also, the auditor should avoid reaching conclusions only by looking at specific, individual complaints – these should be put in the context of their overall impact on the QMS.
How reliable is the information?
In an ideal world, the organisation would monitor the perceptions of all customers, but the costs of doing so m might be prohibitive. Therefore it is necessary to verify the criteria the organisation has used for any sampling of its customers, to ensure it is representative, and reflects the risks both to the organisation and its customers.
The auditor should seek to verify the information provided by comparing with other evidence obtained during the course of the audit (see 3.1). In some cases it may be appropriate for the auditor to verify information directly with the organisation’s customers, though a certain diplomacy will be required when doing this.
How is the data analysed?
Simply collecting data on customer perceptions is not sufficient – the auditor must follow the process through, to check how the data is analysed (see ISO 9001:2000 clause 8.4), and what conclusions are made with respect to the effectiveness of the QMS. Are there any trends? Is the situation stable, improving, or deteriorating? Are customer needs and expectations changing?
Although it is not a requirement of ISO 9001:2000, it may be appropriate to ask the organisation about industry comparisons, or benchmarking activities, to put customer feedback into perspective.
How does the information generated by this process feedback into the QMS as a whole?
Organisations should be using the customer feedback process to trigger corrective and preventive actions, and as one of the overall measures of the QMS’s performance. The way in which these processes interact should also be subject to audit. The auditor should be able to recognise that the output from the customer feedback process forms an important input into other QMS processes, such as data analysis, management review and continual improvement processes. A good auditor will try to ensure the organisation recognises the benefits a sound customer feedback process can bring, and will encourage (but cannot require) the organisation to think beyond simply “meeting the requirements of the standard”.
What are the links to other QMS processes?
The auditor should recognise that the customer feedback process has links with other QMS processes that include, but are not limited to, these clauses:
- 5.6 Management review
- 7.5.2 Process validation
- 7.2.3 Customer communication
- 7.3.6 Design and development validation
- 7.3.7 Design and development changes.