An Evaluation of the UNDP-World Bank Water and Sanitation Program

Version française du résumé

Executive Summary

The UNDP-World Bank Water and Sanitation Program is a continuation of a decades-long effort to improve poor people's access to safe water and sanitation. This document reports the results of an examination of the Program carried out in 1995 by a team of independent evaluators who have analyzed its achievements and shortcomings, and then made recommendations for changes. The team has also attempted to draw lessons from the Program's activities that will benefit the now-forming Global Water Partnership into which the Program will be incorporated.

The origins of the Water and Sanitation Program go back to 1978 when the UNDP and the Bank began to work together to promote low-cost sanitation and water supply solutions for poor people in rural and marginalized urban areas. In subsequent years, through a series of programs and international meetings, emphasis shifted from technical research on equipment such as hand pumps to broader concern about how to get the job done with community participation. This led to a realization that the task could not be accomplished without an effective policy framework, planning and project preparation and that the Program had a role to play in assisting communities and national governments in these areas. Along with the obvious need to consolidate expertise and experience scattered among various programs, these trends led to a 1987 merger of a number of separate projects into a single Water and Sanitation Program.

Since its formation, the Program's mission has evolved and can now be stated as "creating capacity so they can do it themselves." Its focus remains the poor in rural and marginalized urban areas. However, it has increasingly structured projects to support investments in the water and sanitation sector by helping communities and governments develop their capacity to solve problems, to treat water as an economic and a social good and to involve all of the stakeholders in the selection, operation and maintenance of systems. The Program has attracted widespread support and has made major contributions in many countries. The Evaluation Team recommends without qualification that the Program be continued and serve as a major part of the Global Water Partnership. But, there are several areas of concern and several changes that should be considered.

At the outset of its work the team discovered two significant problems. First, the objectives that have been set for the Program are exceptionally broad and ambitious. With a small staff and dwindling financial resources, the Program is being asked to do too much with too little. Just as important, there is not a clear definition of how the Program is to be measured, how its management or sponsors can demonstrate that any of its objectives are, or are not, being reached. As the Evaluation Team's work progressed through meetings, visits to nine countries, and a questionnaire completed by a large number of people involved in the water and sanitation sector, other themes emerged. For example, although there are important exceptions, the management and staff of the Program are not doing an adequate job of transferring lessons from one community to another. At the same time the Program staff is perceived to be isolated; it has seldom looked outside to learn from the approaches and techniques of others involved in similar activities.

In an effort to measure a program that does not have its own ways of measuring the success of its mission, the Evaluation Team used a variety of Program documents to establish five statements which describe the desired situation at the end of the project, and then made an effort to measure progress toward each of these. The Program's goals and the sum of the Team's conclusions are:

1. National and local capacity from community level to the government ministerial level will have been strengthened. Measures to build capacity include extension of participatory training and skills, including greater attention to gender issues; creation and support of national training networks linked into regional and interregional networks; and recruitment of national staff for in-country and regional posts. In many countries the Program has strengthened national and local capacities in the water and sanitation sector. But, there often remains a gap between intent and implementation. Decisions about projects are made by donors and government officials who do not know what the community wants or needs. Responsibilities of ministries and agencies are frequently conflicting and overlapping, a situation that project managers sometimes attempt to remedy by proposing a parallel structure of nongovernmental organizations (NGOs). The Program must review its efforts to build capacity at all levels, to develop and disseminate models that demonstrate how to increase the ability of users to take care of systems with a minimum of external help. It also should increase its emphasis on involving the entire community, especially women, on whom the heaviest burden of inadequate facilities often falls.

2. Participating countries will have made significant progress in shifting sector policies and strategies away from government-driven, top-down approaches towards decentralized, demand-driven, bottom-up approaches that legitimatize a variety of options for provision of services to the poor. The Program's workplans for 1991-95 include projects to help develop policy in 16 countries, strategy in 22 countries, and actual plans in 24. The Program has successfully accomplished much of this. It has provided policy development and planning assistance on national, regional and municipal levels, and has helped plan missions targeted at specific issues, such as investment or the involvement of women. But, adoption and implementation of these plans has varied widely. In some cases, including Ghana and Benin, policies have been implemented. In Pakistan implementation is slow. The Program must identify the reasons for the gaps or lags between plans and action. It should also develop a group of case studies that illustrate how a country or community has accomplished its goal. And it should sponsor tours so that politicians and sector administrators from a country that does not have a working policy framework can visit and learn from a country that does.

3. Qualitative improvements will have been made in the design and implementation of large-scale, sustainable, investment projects targeting the rural and urban poor, funded by the World Bank, the regional development banks and other donors, as a result of Program inputs. Although there have been missed opportunities and a fairly general failure to attract investment from private enterprises, the Program has had a positive effect on investments in the sector. It has been invaluable in assisting in donor coordination and collaboration and has influenced the approach taken to project design by the World Bank and others. It has developed new approaches to community financing schemes and developed strategies that have generated new investment. But, the Program can play a more positive role in coordinating and streamlining plans for projects. Some bilateral donors believe that the Regional Water and Sanitation Groups (RWSGs) through which the Program carries out much of its work, are not sufficiently linked to their programs or the programs of other donors. The Evaluation Team believes that the Program must play a more focused and strategic role, that this might increase the rate of investment. It should give greater priority to micro-level preparation of investment programs: proposals, financing schemes and pilot and demonstration projects. And it should advocate financing of demonstration programs by the Bank and the regional development banks since such programs are essential precursors to larger, more expensive projects.

4. A systematic learning process for testing new approaches to sector development, monitoring the results and feeding them back into the sector development process will have been developed and institutionalized in most of the participating countries. This is the weakest element of the Program. There is little focus on systematic learning, identifying topics and questions to be studied during an on-going project to learn and improve on the project. There is also resistance to studying and drawing lessons from the work of others. At the same time the Program has not used its information staff properly, and budget cuts have further hampered the publication and distribution of information. Lessons learned from projects have not been communicated broadly. Even when publications on the transfer of replicable models have been produced, there has been no systematic effort to learn whether the material is useful or used effectively.

5. Sector coordination will have been significantly improved. The Program has had a positive effect on sector coordination within countries by involving and working with agencies and organizations on specific projects. Generally viewed as an independent organization with a special relationship with the World Bank, it can serve as a channel for informal and non-threatening communication between government agencies on one side and community organizations and private enterprises on the other. But, it has not exploited this advantage as much as it could. It has not often initiated coordination activities beyond the country level. And several government agencies and non-government groups complain that it does not involve them or collaborate with them sufficiently. Among other steps, the Program should take a stronger lead in initiating and developing coordination and communications among all the players in the sector, especially in connection with the planned establishment of regional organizations within the Global Water Partnership. This initiative could include clearly defining its own role in the sector, thus encouraging other agencies, including UNICEF, WHO and the regional banks to define their respective roles.

Management staffing and style

Part of the job of the Evaluation Team was to look at the way in which the program is managed, staffed and structured. The team found a shortage of staff at headquarters and in some regions, signs of low morale, high employee turnover and a need for a full review of management policies, procedures and style. If the Program is to be the flagship of the new Global Partnership, it needs strengthening as quickly as possible. The team has made a number of recommendations in this area. A few are:

- The Program had no current mission statement except for one developed by the Evaluation Team itself. One must be developed and agreed upon.

- There have not been reviews, even at the end of the fiscal year, to evaluate progress against work plans. Reviews of performance against goals should be held at least semi- annually, if not quarterly.

- Means of measuring whether Program activities are making a difference in the lives of the disadvantaged must be developed and promulgated.

- Personnel policies, now set according to Bank rules, as well as work methods and procedures should be designed to be more suitable to the work and style of the Program.

- Opportunities for career development of staff members must be improved.

- Regional groups should be strengthened, perhaps made centers for specialized sector expertise, and given some responsibility for fund raising and liaison with donors in their region.

- Staffing, structure and systems should flow from strategy and priorities.

Money: declining support

Funding for the Program peaked in 1991 and then declined as disbursements from UNDP dropped by nearly 50 percent. Although there has been some increase in the flow of funds from bilateral donors, the loss has not been offset. In 1995, out of funding of nearly $11.1 million, bilateral donors, now the Program's chief support, contributed $6.4 million. The UNDP provided $3.7 million. Aside from reimbursement for services provided, the World Bank contributed $355,000, only 3.2 percent of the total. The financial situation has been further complicated by ties between UNDP funding and individual regional or interregional projects and, especially, by the tendency of bilateral donors to support specific projects rather than core or headquarters activities, which are now largely underwritten by the Bank's relatively small and UNDP's diminishing contributions.

Program managers tried to keep expenditures in line, but they did not cut back activities fast enough or far enough to bridge a funding gap. By 1995, the Program faced a substantial shortfall in its budget for the current year. Mainly as a result of new commitments of support from a few donors, the program was able to close the gap so that it will be able to continue its activities at their present level until the end of 1996.

In addition to the obvious restrictions on activities, financing problems have contributed to low staff morale, high turnover and a widespread perception among the staff that the Program lacks Bank commitment. Among other difficulties, individuals frequently have to be supported with relatively short-term project funds. Some staff members are never certain whether they will continue to be employed. One other serious consequence is the burden on management time. As much as half of the time of key managers is spent on activities related to fund raising.

The Program's financing problems will also be faced by the Global Water Partnership. Partnership planners have suggested that it funds its activities through a water fund that would be at arm's length from any specific donor and able to attract funding from different sources. The Partnership, however, will not be established until August 1996. In the meantime the Program must find funding. The Team proposes that it consider an adaptation of the water fund concept, with the size based on a strategy agreed on by all donors and a share set aside for the core costs associated with the global learning program. Given the impact of the Program on the capacity of countries to carry out and sustain large projects of the type usually funded by the Bank, the team suggests that the World Bank play a major role in raising the financial support for the Program.

A unique function

In sum, the Evaluation Team concluded that the Program performs a unique function. It is properly focused on providing the world's poor in rural and marginalized urban areas with water and sanitation services. It is trusted by nearly all of its partners as an independent entity that is still close enough to the Bank to influence its decisions. It is accessible and credible. Improvements are possible. In some areas they are critical. But the Program should be continued.

Version française du résumé

You have comments or questions?

Send mail to Lucie Bellemare.

Last updated December 16, 1997

1