That confidence is reinforced when the competence of the certification body is attested by an accreditation body – typically there is only one recognized accreditation body per country. Accreditation bodies may in turn choose to participate in multilateral recognition arrangements, under the coordination of the International Accreditation Forum (IAF), to facilitate international trade.

Is certification the only goal?

However, in recent years there has been some debate in Asian developing economies, and elsewhere, about the effectiveness of accredited certification. The question is whether the focus has shifted from one in which organizations strived to develop an effective QMS that could subsequently be certified, to one in which the achievement of certification is the only goal, with a tendency to cut corners to achieve that aim.

It was for this reason that, in 2009, the United Nations Industrial Development Organization (UNIDO) initiated its TE/RAS/09/003 project – Implementation of ISO 9001 Quality Management Systems in Asian developing countries – A survey covering system development, certification, accreditation and economic benefits, funded by the Norwegian Agency for Development Cooperation (Norad), with technical inputs from ISO and the IAF.

Project aims and methodology

The aim of the project was to assess the effect of ISO 9001 certification on the performance of certified organizations, its benefits for their customers and, ultimately, its impact on consumers.

Focusing on “ business-to-business ” transactions in manufacturing and construction industries, the project comprised the following key phases:

  • A survey of 429 major institutional purchasers in 10 Asian developing countries
  • A survey of 604 ISO 9001 certified organizations in 12 Asian developing countries
  • Face-to-face interviews with a small sample of selected purchasers who had demonstrated their understanding of certification and accreditation, and who were able to provide detailed feedback about the performance of their ISO 9001-certified suppliers
  • A series of one-day “market surveillance” visits to 561 certified organizations.

The 12 countries included in the project were: Bangladesh, Bhutan, India, Indonesia, Malaysia, Maldives, Nepal, Pakistan, Philippines, Sri Lanka, Thailand and Vietnam.

Clear economic benefits

The project demonstrated that effective ISO 9001 implementation and accredited certification brings clear economic benefits to the manufacturing sectors of these Asian developing countries. The project revealed the following findings :

Money well spent

98 % of the certified organizations surveyed considered ISO 9001 implementation and certification to have been a “good” (73 %) or “very good” (25 %) investment. Benefits for the organizations included a better understanding of internal processes, more confidence in their ability to produce conforming products, and continual reductions in rework and waste.

Confidence in ISO 9001-certified suppliers

Major institutional purchasers, on the whole, saw accredited ISO 9001 certification as a good way to generate confidence in their suppliers, though the majority of purchasers and certified organizations had only a vague understanding of the role of accreditation. These purchasers expressed good levels of satisfaction in a number of supplier performance parameters (see Figure 1), with ISO 9001-certified organizations consistently outperforming non-certified organizations (see Figure 2).

Notable were the high levels of satisfaction with the intrinsic quality of the goods purchased from ISO 9001-certified suppliers, but the results also showed that responsiveness to customer complaints was a cause for concern. It is recommended that organizations, consultants and certification body auditors pay greater attention to this issue during QMS implementation and assessment, and consult ISO 10002:2004, Quality management – Customer satisfaction – Guidelines for complaints handling in organizations.

Confidence in the organization’s QMS

The 561 one-day “market surveillance” visits to certified organizations were aimed at determining confidence levels related to various aspects of the organization’s QMS, including top management commitment, internal communication, understanding and implementation of the “process approach”, use of the “Plan-Do-Check-Act” cycle to manage processes, internal audits, management review and the overall level of confidence in the certification process.

The visits were carried out by a team of 28 UNIDO-appointed consultants throughout the region, each having undergone a project-specific five-day training course and undertaken a calibration assessment with a group of peers. The full checklist covered 26 topics related to the performance of the organization’s QMS (though intentionally not a repetition of ISO 9001 requirements). The visits were not intended to be “repeat audits” and were not focused on “conforming/nonconforming” outcomes.

Satisfaction guaranteed

Overall, the performance of the 561 organizations that were visited was good and demonstrated the effectiveness of the accredited certification process, with only a small percentage of the certified organizations showing unsatisfactory results. All the consultants were able to discriminate between scores of 1 to 5 within a scale of confidence levels defined for each parameter, as follows :

  • Grade 1 = “Little or no confidence”
  • Grade 2 = “Some evidence presented, but not at all convincing
  • Grade 3 = “OK – no reason to doubt that this is being addressed correctly"
  • Grade 4 = “Clear evidence that this is being done and meets the intent of ISO 9001”
  • Grade 5 = “We can be proud to use this organization as a benchmark for this topic”.

Judgements were made based on each individual consultant’s experience, and were necessarily subjective in nature, although good levels of uniformity had been achieved among the 28 consultants during the training and calibration visits.

Transparency (or the lack thereof) When interpreting the visit results, it is important to remember that the sample of certified organizations was based on “those who were willing to participate in the project”, since it was not possible to make the market surveillance visits mandatory.

However, one major concern identified was a lack of transparency among some certification bodies, and a reluctance (including a refusal in some cases) to make their directory of certified clients available in accordance with Clause 8.3 of the accreditation standard ISO/IEC 17021. This lack of transparency is currently being addressed within both ISO and the IAF. One initiative under consideration is the development of a global directory of ISO 9001-certified organizations.

Some variations in performance were also noted between different certification and accreditation bodies, and in particular, concerns were raised about the performance of certification bodies working under “franchisee” arrangements, as compared with local certification bodies and local branches of multinational certification bodies.

“The customer is always right !”

One important finding was the excellent correlation between the results of the one-day “market surveillance” visits to certified organizations and their customers’ perception of their performance (as determined during the in-depth interviews with purchasing organizations).

Certified organizations that purchasers had ranked among their best suppliers consistently scored better on a wide range of parameters during the one-day visits. Likewise, those identified as unsatisfactory by purchasers had consistently lower scores. This is expected to form the basis for a new “ market surveillance ” approach to monitoring the effectiveness of accredited certification, in line with the ISO/IAF strategic imperative that “output matters”.

Proof beyond doubt

The results of the project demonstrate the benefits of QMS implementation and accredited ISO 9001 certification, proving beyond doubt that ISO 9001 implementation is a good investment of resources. Some areas of concern regarding certification and accreditation practices highlighted during the project are currently being investigated by ISO and the IAF, in order to introduce the appropriate improvements.

Nigel H. Croft
Nigel H. Croft
ISO/TC 176/SC 2, Quality systems

Dr. Nigel H. Croft has been involved in quality management and conformity assessment for over 35 years. He is currently Chair of ISO subcommittee ISO/TC 176/SC 2, Quality systems, responsible for developing ISO 9001 and ISO 9004, and was International Lead Consultant for UNIDO project TE/RAS/09/003 from 2009 to 2011.