In evaluating the appropriateness of the choices made to evolve the Acacia III strategies from what was outlined in the Acacia Prospectus 2006-2011, the panel considered four key variables: (i) the priority areas of work and other choices made against Acacia’s mission as articulated in the 2006-2011 prospectus; (ii) Acacia’s implicit, and later explicitly articulated change logic; (iii) key developments in the ICT4D field; and (iv) constraints and potential risks to relevance, effectiveness and impact.
The panel identified two main drivers for the priorities and choices made during implementation of Acacia III:
These drivers were critical to the program’s relevance, relative coherence and effectiveness. The review panel finds the Acacia III strategy to be well conceived and defensible. The strategy is reflected in the project portfolio for each thematic pillar and is evident in the considerable synergy between the desired and achieved outcomes. Resource allocations were more or less equal to each theme (Annex 10), and the priority areas of work closely aligned with what was originally intended. Acacia’s ‘theory of change’ which was made explicit at the 2009 Acacia Research and Learning Forum (ARLF) shows few signs of departure from the initial strategy aside from some expanded details and updated assumptions about the change logic.5
Few implementation changes are explicitly mentioned in the FPR. The most significant of these is the identification of ‘contributing to a formal body of knowledge’ and ‘applying meaningful gender analysis’ as formal program objectives. This modification does not appear to have affected the overall program direction6 given that the importance of these categories of activity had already been recognized in the Acacia II evaluation findings7 and were included as program activities in the 2006-2011 Prospectus. A related decision was the shift from identifying ‘thriving research networks’ as an objective to recognizing it as an implementation modality. From a program design perspective, this decision was appropriate.
Despite the potential for several of the implementation choices to expose Acacia to risk and programming tensions, the panel finds the rationales underpinning these choices to be generally convincing,8 and notes that they appear to have been managed in a manner that minimized potentially negative impacts on Acacia’s effectiveness:
In verifying and assessing the coherence of the Acacia III program,12 the panel finds significant horizontal (thematic) coherence. The themes, sub-themes and networks consist of projects that complement and build upon one another. In this sense, Acacia can be considered to be ‘more than a sum of its parts.’ There were also many positive comments on the usefulness of the ARLF in October 2009 where network members met together for the first time, enhancing the opportunity to exchange and understand what value can be added through collaboration. (Several key informants confirmed that they were unaware of the potential until this event.) This conclusion, however, is somewhat offset by the seemingly limited cross-fertilization across some networks and projects. The panel recognizes the several efforts made to link the work of different networks but notes that despite the presence in some cases of structures to facilitate cross-fertilization (e.g. emphasis on and promotion of outcome mapping), the information emerging from a number of interviews suggests that these mechanisms did not always work as well as desired.13 This is view is reaffirmed by the comments on page 23 of the FPR which note that there should be more cross-collaboration between gender and sector teams, and that initial efforts to establish cross-fertilization with RIA, for example, were not yet successful.
The panel finds that there were two key success factors in establishing coherence. The first is the well-reasoned strategy for Acacia III. The second is the decision to work within sub-themes with networks and projects that either address a specific challenge through a number of different activities, or provide for comparative work across countries and regions.
The ability to manage for coherence appears to have been facilitated by the presence of three important factors. First, a well-coordinated stable team of experienced and specialized Program Officers and network leaders; second, a shared understanding of Acacia’s strategy and operational principles; third, a good Program Officer-to-budget ratio that facilitates their ongoing engagement in shaping the area’s work.
Overall, the review panel finds that Acacia III has been managed in a manner which ensured that program implementation was in line with what was envisaged at the start of the existing program period. Priorities were established and changes were made in a thoughtful manner, with convincing rationales offered for divergences from what was originally intended.