Evaluation in the United Nations Secretariat

United Nations ST/AI/2021/3 Secretariat 6 August 2021 21-11077 (E) 160821 *2111077* Administrative instruction Evaluation in the United Nations Secretariat The Under-Secretary-General for Management Strategy, Policy and Compliance, pursuant to section 4.2 of Secretary-General’s bulletin ST/SGB/2009/4 and for the purpose of prescribing instructions and procedures for the implementation of article VII, Evaluation, of the Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation (ST/SGB/2018/3) promulgates the following: Section 1 Purpose and scope 1.1 The present instruction describes the Organization’s requirements and procedures for the management, conduct and use of evaluations and sets out the operational guidelines for article VII, Evaluation, of the Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation. It puts into effect the Secretary-General’s intention to strengthen the evaluation capacity of the Secretariat so as to better inform programme planning and reporting on programme performance, among other proposals contained in his report on shifting the management paradigm in the United Nations (A/72/492), which the General Assembly took note of in its resolution 72/266 A. 1.2 In view of the mandates of the Inspection and Evaluation Division of the Office of Internal Oversight Services (OIOS), as set forth in Secretary-General’s bulletins ST/SGB/273 and ST/SGB/2002/7, and its role as the central evaluation unit referred to in ST/SGB/2018/3, the present instruction clarifies the architecture of the evaluation function in the Secretariat. It also outlines the governance arrangements and accountabilities, as well as performance standards, for evaluation functions in the Secretariat. 1.3 Evaluation is a shared function across Secretariat entities. The Secretariat entities are the departments and offices, including offices away from Headquarters, special political and peacekeeping missions, regional commissions, resident or regional coordinators’ offices and any other unit tasked with programmed activities. 1.4 The Secretary-General’s report (A/72/492) and related resolutions and reports refer to evaluation undertaken by both oversight bodies and entities and to self - evaluation as referenced and defined in section VII of ST/SGB/2018/3. The term ST/AI/2021/3 21-11077 2/6 “evaluation” is used in the present instruction in reference to both. The provisions of the instruction apply to all Secretariat entities. Accordingly, all entities are required to have an evaluation policy that is consistent with the present instruction. Forthcoming guidance will address transitional measures. Section 2 Evaluation procedures Evaluation planning and coverage norms 2.1 Each entity must develop an annual evaluation plan and ensure adequate allocation of funds for evaluation capacity to deliver the plan. The results of evaluations and lessons learned must be considered in the preparation of plans and budgets for future cycles and reported in the budget documents. 2.2 Secretariat entities undertake various types of evaluation, including programme, thematic and cross-cutting evaluations. The types of evaluation activity undertaken by each entity depends on its role, mandate and evaluation capacity. The Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation note the need for all programmes and subprogrammes to be evaluated on a regular, periodic basis, irrespective of the source of funding. 2.3 The Inspection and Evaluation Division of OIOS, as the central evaluation unit, conducts its programme and subprogramme evaluations at least once every eight years, as specified in the table below. Specifically, the types of evaluation and evaluative activities and their coverage norms are as follows: (a) Programme evaluations assess the overall relevance, effectiveness, impact and efficiency of a single Secretariat programme or subprogramme or of a peacekeeping operation or special political mission. As endorsed by the General Assembly,1 each Secretariat programme is normally evaluated at least once every eight years; (b) Thematic evaluations assess high-risk cross-cutting themes or activities across multiple Secretariat entities, or the cumulative effects of multiple programmes sharing common objectives and purposes, or the effectiveness of coordination and cooperation among different entities. There is normally at least one thematic evaluation on a cross-cutting theme among the priority areas of work of the Organization each biennium; (c) Inspections are short, focused and targeted reviews of an organizational unit, issue or practice perceived to be of high risk, in order to determine the extent to which it adheres to established norms, good practices or other pre-determined criteria, and to identify corrective action as needed. These are ad hoc and conducted as and when identified through regular risk assessments; (d) Reports on strengthening the role of evaluation and the application of evaluation findings on programme design, delivery and policy directives together with the evaluation dashboard are the main products through which the Inspection and Evaluation Division seeks to strengthen evaluation capacity within the Secretariat. The reports are produced biennially (ST/SGB/2018/3, regulation 7.4); __________________ 1 In its resolution 65/244 the General Assembly endorsed the recommendation of the Committee for Programme and Coordination at its fiftieth session (A/65/16) to approve the proposal of OIOS for a cycle of eight years to cover all Secretariat programmes so as to improve the programme evaluation cycle (A/65/6 (Prog. 25), para. 25.6). ST/AI/2021/3 3/6 21-11077 (e) Triennial reviews are assessments undertaken three years after a General Assembly mandated inspection or evaluation to review the status of implement ation of report recommendations. All such mandated inspections and evaluations are subject to triennial reviews (A/37/38, para. 362). 2.4 All entities are subject to the requirement that all programmed activit ies shall be evaluated by the entity itself, by OIOS or by an external entity such as the Joint Inspection Unit and other external bodies as approved by the General Assembly, as specified in regulations 7.2 and 7.3 of the Rules and Regulations. Evaluation coverage norms and criteria for consideration in establishing the periodicity of evaluations are outlined in the table below. As such, Secretariat entities shall establish clearly, within their respective evaluation policies, the specific regular periodic basis within which all programmes and subprogrammes under their responsibility are evaluated. OIOS and the Department of Management Strategy, Policy and Compliance shall provide guidance on evaluation planning to ensure coordination and adequate evaluation coverage of the Secretariat. Evaluation coverage norms in the Secretariat Evaluation Frequency Evaluation manager Quality assurance Accountability for management response and use of evaluation results Accountability for monitoring implementation of recommendations Programme/ subprogramme evaluation by OIOS At least once every eight years and as requested by intergovernmental bodies OIOS Inspection and Evaluation Division OIOS Inspection and Evaluation Division Entities OIOS Programme/ subprogramme evaluation by entities At least once every six years Heads of entities or designated evaluation managers Heads of entities or designated evaluation managers with support from OIOS as needed Entity and programme/ subprogramme management Entity head of evaluation or designated focal point Other (project, thematic area, regional or country operation, etc.) As necessary, based on risk assessment and/or specific donor requirements or entity-specific evaluation policy Heads of entities or designated evaluation managers and project managers as appropriate Heads of entities or designated evaluation managers and project managers with support from OIOS as needed Entity programme/ subprogramme and project management Entity programme/ subprogramme and project management Management and conduct of evaluations 2.5 Entities shall design, conduct and manage their evaluations in line with the regulations and rules set out in the Secretary-General’s bulletin (ST/SGB/2018/3) and the present instruction. The norms and standards for evaluation of the United Nations Evaluation Group provide additional non-binding advice on the conduct and management of evaluations. 2.6 Evaluations shall follow key minimum parameters to ensure useful and credible results: ST/AI/2021/3 21-11077 4/6 (a) Management arrangements for each evaluation shall ensure impartiality and independence. The designated evaluation manager and team members shall not be part of the team that designs or manages the implementation of the assessed policy, plan or programme; (b) Entities shall develop an inception report or terms of reference that will contain an evaluation objective, evaluation questions and evaluation design and methods; (c) Evaluation results, lessons learned and recommendations shall be incorporated and reflected in planning and budget documents so as to inform strategic and transparent programme delivery (as specified in ST/SGB/2018/3, rule 107.4). Quality assurance 2.7 The Inspection and Evaluation Division of OIOS, in consultation with the Business Transformation and Accountability Division of the Department of Management Strategy, Policy and Compliance, develops and provides guidance and tools to support the conduct of evaluations. 2.8 Heads of entities are responsible for the existence of an appropriate quality assurance system in their respective entities. The quality assurance system shall include a clear process, with specific and transparent standards, for assessing the overall quality of evaluation using agreed professional standards for the conduct of evaluation. OIOS and the Department of Management Strategy, Policy and Compliance will provide guidance on minimum quality standards. When Secretariat entities need assistance in conducting quality assurance at key points in the conduct of an evaluation, they may request such assistance from the Inspection and Evaluation Division of OIOS. Key points include identifying evaluation resources such as consultants; preparing terms of reference, including developing potential methodology; during the inception phase when finalizing the evaluation design; at the time of the draft inception report; and at the time of the draft evaluation report. Section 3 Use of evaluation data, findings and follow-up 3.1 Heads of entities shall promote the use of evaluation data, findings and follow - up through an interactive process that involves programme managers in the entity being evaluated, end users and other relevant officials. Heads of entities shall also ensure that recommendations, findings and lessons learned are considered in planning and in the preparation of budgets for future cycles and are integrated into policies an d programmes. Management response to evaluation recommendations 3.2 Management in all entities is required to prepare a response to each evaluation. Management responses to recommendations shall be discussed with programme managers and include specific, time-bound actions. In instances where Management decides to adopt recommendations made pursuant to an evaluation, it shall do so by clearly assigning responsibilities to units that will implement the recommendations. Management responses shall be made public, except for circumstances in which such response would divulge confidential information. ST/AI/2021/3 5/6 21-11077 Section 4 Complementarity with other learning and oversight functions 4.1 The Inspection and Evaluation Division of OIOS and the evaluation units of Secretariat entities rely on data and information from other functions such as programme and performance monitoring, research and audit. While maintaining independence, the Division and Secretariat evaluation units coordinate with internal and external oversight bodies such as the Internal Audit Division of OIOS, the Board of Auditors and the Joint Inspection Unit to ensure that evaluations can take advantage of efficiencies in the generation and use of data and evidence for assessing organizational effectiveness. 4.2 Secretariat entities may request the Inspection and Evaluation Division of OIOS and the Department of Management Strategy, Policy and Compliance to support the utilization of relevant performance monitoring systems and data for evaluations and facilitate the sharing of information and lessons learned. Section 5 Provisions on roles and responsibilities of the Evaluation Management Committee, the Evaluation Advisory Committee and Secretariat entities Evaluation Management Committee 5.1 The Evaluation Management Committee will be a subcommittee of the Secretary-General’s Management Committee, chaired by the Under-Secretary- General for Management Strategy, Policy and Compliance. The Business Transformation and Accountability Division of the Department of Management Strategy, Policy and Compliance will serve as the secretariat to assist the Committee in exercising its mandate. 5.2 The Evaluation Management Committee advises the Secretary-General (or designate) on issues relating to the planning, budgeting and policies related to evaluation. It is intended to foster demand and use of evaluation within the Organization, to raise awareness of the added value of evaluation and to ensure that accountabilities for uptake and use of evaluation results are exercised, including management response and follow-up on the implementation of recommendations. Evaluation Advisory Committee 5.3 The Evaluation Advisory Committee is an independent external body, appointed by the Under-Secretary-General for Internal Oversight Services to advise the Director of the Inspection and Evaluation Division on issues relating to the evaluation function, including implementation of the evaluation policy; risk-based planning of evaluations; recommending improvements to evaluation methodologies, procedures, quality assurance mechanisms and management response; and ways to enhance the use of evaluations. The Business Transformation and Accountability Division has observer status in the Evaluation Advisory Committee. Secretariat entities 5.4 Evaluation is an integral part of the management of the programme planning and budgetary cycle. Each head of entity is responsible for ensuring the establishment of the entity’s evaluation policy and an adequately resourced evaluation capacity. 5.5 All heads of Secretariat entities shall: (a) Ensure that the conduct of evaluation is functionally independent within the entity; ST/AI/2021/3 21-11077 6/6 (b) Ensure the integration of respect for gender equality and disability inclusion in evaluation procedures and practices; (c) Use qualified external and internal evaluators with relevant professional competencies and experience, including evaluators from the roster developed and maintained by the Inspection and Evaluation Division where in-house capacity is lacking; (d) Ensure that appropriate mechanisms are in place for management to monitor and respond to evaluation recommendations, including outlining concrete actions that are to be undertaken, and to follow up on recommendation implementation; (e) Conduct joint evaluations with other Secretariat entities on cross -cutting areas, including those that contribute to the implementation of the Sustainab le Development Goals, with coordination by the Inspection and Evaluation Division as required. Section 6 Resources 6.1 Heads of entities shall ensure adequate capacity and resources to manage and conduct evaluations and provide effective quality assurance. Heads of entities shall also ensure that there is adequate capacity to deliver these key roles. Some of these services may be provided directly by the entity itself, or they may be provided through other arrangements such as consultancies or long-term agreements with centres of excellence.2 6.2 Secretariat entities shall take into consideration the general view of the Joint Inspection Unit that funding for evaluation activities should fall within the range of 0.5 to 3 per cent of entity expenditure, depending on the mandate of the entity, its size, types of evaluation undertaken and the role of evaluation in the entity (JIU/REP/2014/6, para. 77). Section 7 Implementation, reporting and review 7.1 The present instruction will come into force at the time of its issuance. 7.2 In accordance with rules 107.2 and 107.3 of the Regulations and Rules Governing Programme Planning, the Programme Aspects of the Budget, the Monitoring of Implementation and the Methods of Evaluation, the Inspection and Evaluation Division of OIOS, as the central evaluation unit, will provide further guidance, with support from the Business Transformation and Accountability Division of the Department o f Management Strategy, Policy and Compliance, on the implementation of the present instruction. Nothing in the instruction shall be construed as limiting the operational independence of OIOS with regard to its mandated activities. 7.3 The instruction will be reviewed two years from the date of issuance. (Signed) Catherine Pollard Under-Secretary-General for Management Strategy, Policy and Compliance __________________ 2 Centres of excellence include research institutions, academic institutions and the like that have evaluation expertise and could provide evaluation services to Secretariat entities.