Quality assurance in higher education

• Summary
• The audit
• Findings
• Recommendations
• Conclusion
• Responses of the minister, the VSNU and the Higher Professional Education Council
• Afterword by the Court of Audit

Summary

Higher education in the Netherlands comprises university education (WO) and higher professional education (HBO). Since 1985, higher education institutions have been allowed greater administrative autonomy, offset by the obligation to answer for the quality of the education they provide. A quality assurance system has been introduced, which the educational institutions were themselves to be responsible for operating. This system consists of a regular cycle of self-evaluation by the institutions and subsequent appraisal by a review committee composed of external experts. The reports of the review committees are in the public domain. The entire system is organised and coordinated by the Association of Dutch Universities (VSNU) and the Higher Professional Education Council. The Education Inspectorate supervises the system on behalf of the minister.

Up

The audit

The Netherlands Court of Audit investigated:

whether the external quality assurance system fulfils its purpose of enabling the Minister of Education, Culture and Science to monitor the quality of higher education. This is something he needs to do in order to fulfil his statutory responsibility for the quality of education and for the use of public funds by the higher education system;

whether in practice the educational institutions implement the recommendations for improvement produced by the external quality assurance system.

Up

Findings

The audit showed that the system of external quality assurance in higher education is an adequate and relatively low-cost way for the minister to ensure that the institutions observe their responsibility for safeguarding the quality of the education they provide. The review committee appraisals give the minister an indirect impression of the quality of individual courses. The Inspectorate assesses the review committees' reports, advises the minister on the basis of the report findings, and supervises the implementation of remedial measures by the educational institutions. The Court of Audit feels that the system enables the minister to fulfil his responsibility for the quality of education and for the use of public funds by the higher education system. The remedial function of the system works well. In the cases investigated, the recommendations in the review committees' reports could be shown to have had a satisfactory effect on the educational process.

Despite its predominantly positive general opinion of the quality assurance system, the Court of Audit has a number of major criticisms of the way the system works.

• The reports of university review committees are difficult to compare. This is true both of reports on similar courses in different institutions and of consecutive reports on a single institution. The Court of Audit feels this is an important point because such comparisons are necessary in order to be able to form an impression of trends in the quality of education over time. The Court of Audit feels it is unfortunate that this cannot be done at present.

• The audit revealed that those educational institutions where the quality assurance system is fairly well established pay less regard to review committee findings. In practice, a black-and-white finding of satisfactory/unsatisfactory provides sufficient incentive for courses on the borderline, but offers no stimulus for further improvement in courses found to be satisfactory.

- The way in which the Inspectorate decides whether educational institutions qualify for supervisory action is a weak link. The Inspectorate assesses the adequacy of the educational institution's responses to the findings in the review committee's report. If it finds the response inadequate, the institution is labelled as giving cause for concern. It is not, however, clear how the findings in the review committee's report and the response of the institution are weighed up to come to this decision. There are no rules to ensure uniform treatment of cases in this regard. It is therefore equally unclear whether these decisions can be compared with each other.

Up

Recommendations

In order to improve the comparability of the review committees' reports, the Court of Audit recommends that the VSNU and Higher Professional Education Council should formulate the definitions and norms more precisely. These should then be carefully applied by the review committees. To increase the stimulus provided by the quality assurance system for the 'better' courses, a more detailed system of findings needs to be introduced. This would also make it easier to compare different courses. The 2000 - 2005 protocol for reviewing university courses and the basic framework for the review of HBO courses already meet both these needs: they include more detailed and concrete norms with scales on which to score courses.

The Inspectorate should take steps to improve the transparency of the way it reaches its decisions. A number of measures have already been taken. This is all the more important because contacts between the Inspectorate and the institutions have become more direct as a result of the new agreements reached between the parties in December 1998.

In addition to these recommendations for the improvement of quality assurance mechanisms, the Court of Audit has several recommendations aimed at improving the way the system is implemented.

Firstly, the Inspectorate should strive to work within the set time-limits, particularly now that the procedures are being further streamlined on the basis of new agreements with the VSNU and the Higher Professional Education Council.

Secondly, the Inspectorate and the VSNU/Higher Professional Education Council should reach agreement on arrangements to ensure complete coverage by reviews. At present, each of the parties thinks that the other is responsible for this, with the result that there is no guarantee that every educational institution that should receive a visit from a review committee actually gets one.

Finally, the Minister of Education should act on his own initiative, rather than, as now, at the request of Parliament, to draw up in response to the findings of the Inspectorate an annual position paper on the quality of higher education and trends within it.

Up

Conclusion

The minister is working to achieve a further increase in the autonomy of educational institutions and to deregulate the education system. This demands a new balance between the institutions' autonomy and their answerability. Against this background, the minister has outlined the revised role and position of the Inspectorate in a policy document entitled 'Variety and safeguards: proposals for the future of educational inspection'. There is to be new legislation laying down the key features of good education to be used as criteria by the Inspectorate. This means that the education system will have access to detailed features, norms and indicators which constitute minimum quality standards for education. At present, no sector of education is excepted from this. In the case of Higher Education, the content of the new legislation will depend in part on the eventual plans for a higher education accreditation system (the development of which was announced in the Higher Education and Research Plan 2000).

The contents of the 'Variety and safeguards' policy document are still subject to decision by the Lower House. The Education Council has also issued an advisory report on the document. The Council feels that no distinction should be drawn between standards of teaching and other quality standards and that all quality standards should in any case have a statutory basis. The fact that the development of the system has led to the gradual raising and clarification of standards over time can, in the opinion of the Court of Audit, be seen as a pointer to the need at any rate to provide a more detailed specification of quality requirements. The Court of Audit feels that this will increase the comparability and transparency of reporting by the Inspectorate and the review committees.

Up

Responses of the minister, the VSNU and the Higher Professional Education Council

The minister has endorsed the conclusions and recommendations of the Court of Audit, feeling that they are consonant with the plans expressed in the Higher Education and Research Plan 2000 for the reform of the system of quality assurance.

The minister has notified the Court of Audit that the recommendations are already being implemented or that action has been taken to start to improve supervision. In response to the recommendation to draw up an annual position paper on quality assurance, the minister says that he prefers to maintain the existing practice and draw up a position paper only when there is reason to do so.

The VSNU was delighted with the Court of Audit's conclusion that the system of external quality assurance is generally working well within university education. In its view, the recommendations were in keeping with the developments already taking place within the system. It was struck by the fact that the audit had paid no attention to the relationship with the system of external quality assurance in the field of university research. It felt, however, that caution should be exercised in establishing minimum quality standards. The VSNU expected that the formulation of explicit criteria, which will also be used to assess standards, will lead review committees to assess the quality of courses in a more uniform way than is currently the case. The VSNU said that it would in future have no further influence on the completeness of the coverage provided by the system because institutions will be able to seek review by bodies other than the VSNU. It thought that it was the Inspectorate's job to ensure that every course was checked.

The response from the Higher Professional Education Council identified a number of factual inaccuracies, which the report has now been revised to correct. Otherwise, the Council wrote that it could not endorse the Court of Audit's conclusion that the system provides no stimulus for further development by courses found to be satisfactory.

Up

Afterword by the Court of Audit

The Court of Audit is delighted with the positive responses to its conclusions and recommendations by the minister, the VSNU and the Higher Professional Education Council.

The minister has not adopted the recommendation to draw up an annual position paper on the quality of higher education. Given that the VSNU and Higher Professional Education Council feel that the new review guidelines will produce greater comparability between reviews and review committee reports, the Court of Audit calls on him to reconsider his decision. If the review committees apply the new guidelines, the minister should be able to use their reports in combination with the findings of the Inspectorate to adopt a regular position on trends in the quality of higher education.

Conclusion	Recommendation	Undertaking
The system enables the minister to fulfil his responsibility for the quality of higher education and the use of public funds
The Netherlands Court of Audit feels there is a need for an annual position paper from the minister concerning trends in the quality of higher education	The minister should, on his own initiative, draw up an annual position paper on the quality of higher education and trends in it	The minister prefers to maintain the current practice
Recommendations in the review committees' reports can be shown to have had a satisfactory effect on the educational process
Review committee reports on university courses are difficult to compare and give no impression of trends in quality	The VSNU and Higher Professional Education Council should produce more precise definitions and norms, which review committees should then apply with care	The introduction of the 2000 - 2005 protocol for reviewing university courses (VSNU) and the basic framework for the review of HBO courses (Higher Professional Education Council) will increase the comparability of review committee reports
For institutions with well-developed quality assurance systems, there is insufficient stimulus for further improvement	A more graduated system of scoring is required	The 2000 - 2005 protocol and basic framework provide for this
The Inspectorate does not make it clear how it decides on supervisory action on the basis of the review committee's report and the institution's response to it	The Inspectorate should take steps to increase the transparency of its assessment procedures	The Inspectorate has recorded its procedures and formats in an internal manual. A frame of reference is being developed for inspectors and other staff. The Inspectorate is to issue reports of the findings on the basis of which it reaches its decisions. A procedure will be introduced to increase the intersubjectivity of project teams.
The Inspectorate does not always meet the agreed time-limits	The Inspectorate should take steps to improve its performance in meeting the agreed time-limits
It is unclear who is supposed to check whether all the institutions are actually reviewed	The Inspectorate, VSNU and Higher Professional Education Council should agree who is responsible for checking coverage	The Inspectorate will report on the completeness of coverage by the review system, beginning with the 1998 annual reports (published in 1999)

Up