External Review of the idrc acacia Program icon

External Review of the idrc acacia Program

страницы:   1   2   3   4   5   6   7   8   9   ...   12

External Review of the IDRC Acacia Program

Final Report

August 31, 2010

By: Daniel Pare, Zenda Ofir and Jonathan Miller

With support from Emily Taylor



Introduction 1

Approach and Methodology 1

Challenges and Limitations 2

Appropriateness of Prospectus Implementation 2

Quality of Research Publications 5

Outcomes 7

Key Issues for the IDRC Board of Governors 13

Concluding Remarks 15

Annex 1: Description of Review Approach and Methodology 16

Annex 2: Review Matrix 21

Annex 3: List of Projects Reviewed (N=34) 25

Annex 4: Documents Reviewed 26

Annex 5: List of Key Informants 30

Annex 6: Interview Protocols for Key Informants 32

1. Interview Guide for Network Leaders & Selected Project Principal Investigators 32

2. Interview Guide for Experts 38

Annex 7: Survey Design and Distribution 42

Annex 8: Details of Research Quality Assessment 46

n/a=0 46

very poor=1 46

Excellent=5 46







^ Annex 9: Citation Research 47

Annex 10: Acacia Financial Data 51

Annex 11: Acacia Research Outputs/Publications by Sub-Theme 53

Annex 12: Acacia Composite Research Outputs/Publications Index 2006-2010 (N=920) 54

Annex 13: Outcomes Analysis 55

Annex 14: Panel Biographies 56

Endnotes 57


ACA2K African Copyright & Access to Knowledge

ANLoc African Network for Localization

ARLF Acacia Research and Learning Forum

AVOIR African Virtual Open Initiatives and Resources

CCK Communications Commission of Kenya

CLDR Unicode Common Locale Data Repository

EFF Electronic Frontier Foundation

FPR Final Program Report

GIS Geographical Information Systems

GRACE Gender Research in Africa into ICTs for Empowerment

ICASA Independent Communications Authority of South Africa

ICT Information and Communication Technology

ICT4D Information and Communication Technology for Development

ITU International Telecommunication Union

NCC Namibian Communications Commission

OASIS Developing Open Architecture, Standards and Information Systems

OECD Organization for Economic Co-operation and Development

PALM Publishing and Alternative Licensing Model of Africa

PanAf Pan African Research Agenda on the Pedagogical Integration of ICTs

PCD Peace, Conflict and Development Program

PDA Personal Digital Assistant

PICTURE Poverty, Information and Communication Technology in Urban and Rural Eastern Africa

RIA Research ICT Africa

rPCR Rolling Program Completion Report

SCA Scholarly Communication Access

UHIN Uganda Health Information Network

MHIN Mozambique Health Information Network

WIPO World Intellectual Property Organization


The period 2006-2011 marks Phase III of the IDRC’s Acacia program, intended “to support research on ICTs that improve livelihood opportunities, enhance social service delivery, and empower citizens while building the capacity of African researchers and research networks.” (Acacia Prospectus 2006-2011, p. 1). The program is structured around three themes:

  • People Empowerment (Gender Research, Digital commons, Localization)

  • Social Service Delivery (Tertiary Research and Education Networking, Local Governance, Health, New Learning Environments and Practices)

  • Economic development and Opportunity (Social and Economic Development, Infrastructure Policy and Indicators, Small Scale Agriculture)

The outcomes of Phase III originally were intended to achieve four high-level objectives: Sustained Policy Dialogue, Thriving Research Networks, Enhanced Research Capacity in ICT4D, and More Social and Technical Innovation in ICTs. By and large these objectives have guided the Acacia team through to the present despite some minor modifications that are discussed in this review.

The Acacia III budget of $64.9m is spread across 161 projects. Over 80 percent of the budget is applied to some 13 networks, with allocations averaging $4.3m, but ranging from $707,000 (African Virtual Open Initiative and Resources (AVOIR)) to $8m (Research ICT Africa (RIA)). Allocations to individual projects average approximately $400,000 and vary widely, from a few thousand dollars to support occasional seminars, to over $2.5m for ongoing support to some networks.

The Acacia External Review Panel commenced work in April 2010 and comprises Jonathan Miller, Zenda Ofir and Daniel Paré, ably assisted by Emily Taylor. This external review of Acacia’s programming follows the revised program evaluation strategy initially used in 2009 for externally reviewing the Peace, Conflict and Development (PCD) Program. Throughout its review process the panel found itself reflecting on the strengths and weaknesses of the new approach. In line with the view expressed by the PCD review panel, we concur that the involvement of the Acacia team in critical reflection as part of the review process is a key component of learning and hope that it has, indeed, shifted the process to “one that is done with the team, rather than ‘to’ the team” (PCD, 2009, p. 1).

Approach and Methodology

This review comprises an external verification of the Acacia team’s self-evaluation of their work over the past five years as reported in the Final Prospectus Report (FPR). The parameters for this exercise were established by four questions stipulated in the Terms of Reference (ToRs) provided to the panel members:

  • To what extent was the implementation of the program’s prospectus appropriate?

  • Overall, was the quality of the research outputs/publications supported by the program acceptable (given the context/intended purpose, etc.)?

  • To what extent are the program’s outcomes relevant, valuable and significant?

  • What are the key issues for the IDRC’s Board of Governors?

The methodology employed by the review panel was guided by an evaluation matrix consisting of 19 questions that elucidate the four key review questions.1 A mixed methods approach using different sources was employed for data collection in order to enable adequate triangulation for credibility of the findings. Key components were (i) a document review of program and selected project documents provided by the Acacia team; (ii) a synthesis of rolling Project Completion Reports (rPCRs) to obtain aggregated information on completed projects above CAD150 000; (iii) an on-line survey distributed to 176 potential respondents with 36 responses received; (iv) a total of forty 1.0-1.5 hour interviews (telephone and face-to-face) with purposefully selected key informants; (v) an assessment of the quality of the research outputs/publications of 34 projects spanning the three thematic pillars of the Acacia program;2and (vi) a citation analysis of Acacia publications. Detailed information about the panel’s approach and methodology is provided in Annexes 1 to 9.

Challenges and Limitations

The review panel faced several challenges and limitations. First, in light of the broad scope of the review and the priority focus on verification of the FPR set out in the Terms of Reference, the panel was not able to interview as many people as it would have liked who are directly involved in some manner with Acacia. Instead, it prioritized understanding the internal approaches, processes and dynamics of the Acacia program. The panel members also are conscious that the review is based on a relatively swift review of primary and secondary sources and the structured gathering of input from a small (yet highly credible) group of experts—most from within Acacia, but a few from outside who all have recognized experience in ICT4D. Given the robustness of the methodology employed, the panel is confident in the conclusions that it has drawn, but stresses that these have not been verified through engagement on the ground with Acacia projects in Africa.

Second, a large number of Acacia’s Phase III projects are ongoing or in the early stages of implementation. It is still too early to assess the potential relevance, significance and value of their eventual research outcomes. Given the large portfolio of Acacia projects, the panel was only able to analyze a limited number of projects in depth. The work of the panel also was constrained by the serious deficiencies in Acacia’s information and knowledge management systems.

Third, the core concepts used in, or at least relevant to, Acacia Phase III (e.g. quality, innovation, coherence, significant outcomes) were not formally defined by the Acacia program. The review panel developed its own understanding of these concepts —informed, in part, by input for Acacia team members—and remains cognizant that its understanding may not fully align with that held by the whole of the Acacia team and its research partners.

Download 0.54 Mb.
leave a comment
Date conversion04.03.2012
Size0.54 Mb.
TypeДокументы, Educational materials
Add document to your blog or website

страницы:   1   2   3   4   5   6   7   8   9   ...   12
Be the first user to rate this..
Your rate:
Place this button on your site:

The database is protected by copyright ©exdat 2000-2017
При копировании материала укажите ссылку
send message