Social Return on Investment for Emergency Obstetric Care Training in


Kenya

Dedication

“To philanthropy and all those contributing to making the world a better place”

List of tables

Table 1‑1: Summary table of social impact assessment methods……

Table 2‑1: WHO emergency obstetric care signal functions………..

Table 3‑1: Krlev et al. 12-point quality assessment framework………

Table 3‑2: Sources of retrieved SROI studies………………….

Table 3‑3: Criterion quality scores of health SROI studies across the 5 dimensions

Table 3‑4: Health interventions that have been measured with SROI…

Table 3‑5: Summary table describing application of the SROI methodology in Health

Table 3‑6: Summary of duration of health SROI analysis………….

Table 4‑1: Outputs of the current phase of the Making it Happen programme

Table 4‑2: Content of the LSS-EOC & NC…………………….

Table 4‑3: Distribution of Kenyan population across age groups…….

Table 4‑4: Kenyan Key development indicators………………..

Table 4‑5: EmONC indicators in Kenya compared to UN standards….

Table 5‑1: Description of SROI principles…………………….

Table 5‑2: MiH outcome and output data indicators relevant for SROI analysis

Table 6‑1: Stakeholder analysis of the EmONC training component of the Making it Happen programme

Table 6‑2: Primary and secondary data required from the included stakeholders

List of figures

Figure 1‑1: Types of Social Return on Investment Studies…………

Figure 2‑1: Annotation of the logic model (version 1)…………….

Figure 2‑2: Annotation of the logic model (version 2)…………….

Figure 2‑3: DFID 3Es + CE framework for Value for Money………..

Figure 3‑1: Schematic representation of the conceptual framework underpinning the review

Figure 3‑2: PRISMA flow diagram summarising the search process….

Figure 3‑3: Mean quality scores of health SROI studies since 2005….

Figure 3‑4: Number of public health SROI articles published since 2005.

Figure 3‑5: Number of health SROI studies published by year, showing countries where the methodology was applied

Figure 3‑6: Distribution of countries conducting SROI studies………

Figure 4‑1: Map of Kenya showing the eight provinces and the surrounding countries

Figure 5‑1: SROI process map……………………………

Figure 5‑2: Methods to be used within the SROI data collection stages..

Figure 6‑1: Analysis of stakeholders related to EmONC training in Kenya

List of appendices

Appendix 1: Search strategy

Appendix 2: Systematic review results

Appendix 3: Ethics

Appendix 4: Materials relating to Making it Happen

Appendix 5: Questionnaires

Abbreviations

ACAFIAtkisson Compass Assessment for Investors

AIDSAcquired Immune Deficiency Syndrome

BACOBest Available Charitable Option

BEmOCBasic Emergency Obstetric Care

BEmONCBasic Emergency Obstetric and Newborn Care

BoP Base of Pyramid

BScBalanced Scorecard

BVABlended Value Accounting

CAQDAS

CEmOCComprehensive Emergency Obstetric Care

CEmONCComprehensive Emergency Obstetric and Newborn Care

CBACost Benefit Analysis

CEACost-Effectiveness Analysis

CMA Cost-Minimization Analysis

CMNHCentre for Maternal Newborn Health

CRTCluster Randomised Trials

CSOCivil Society Organisation

CUACost-Utility Analysis

DALYs Disability-Adjusted Life Years

DFIDDepartment for International Development

DHDepartment of Health

DHS Demographic and Health Survey

EEEconomic Evaluation

EmOC Emergency Obstetric Care

EmONCEmergency Obstetric and Newborn Care

EOC & NCEssential Obstetric Care and Newborn Care

FGFocus Group

FGDFocus Group Discussion

GRIGlobal Reporting Initiative

HCPHealth Care Providers

HEFCEHigher Education Funding Council for England

ICPIntegrated Care and Prevention

IDIIn-Depth Interview

KHSSIP Kenya Health Sector Strategic and Investment Plan

KIIKey Informant Interview

KPIKey Person Interview

LADSILabour and Delivery Satisfaction Index

LATHLiverpool Associates in Tropical Health

LEMLocal Economic Multiplier

LMICsLow and Middle Income Countries

LSS-EOC&NCLife Saving Skills – Essential Obstetric Care and Newborn Care

LSTMLiverpool School of Tropical Medicine

MARPMost At Risk Population

MDG-ScanMillennium Development Goal Scan

MICSMultiple Indicator Cluster Survey

MIFMeasuring Impact Framework

MiHMaking It Happen

MMRMaternal Mortality Ratio

MNHMaternal and Newborn Health

MOHMinistry of Health

MOMSMinistry of Medical Services

MOPHSMinistry of Public Health and Sanitation

MSCMost Significant Change

nefNew Economics Foundation

NGOsNon-Governmental Organisations

OASISOn-going assessment of Social Impacts

OVCOrphan and Vulnerable Children

PLHIVPeople Living with Human Immuno-deficiency Virus

PLHWAPeople Living With AIDS

PSIAPoverty and Social Impact Analysis

PTOPerson Trade-Off

QALYsQuality-Adjusted Life Years

QIQuality Improvement

RCOGRoyal College of Obstetricians and Gynaecologists

RCTRandomised Controlled Trials

REDFRoberts Enterprise Development Fund

RFRoberts Foundation

ROIReturn on Investment

SAASocial Accounting and Auditing

SBASkilled Birth Attendant

SCStakeholder Consultation

SCBASocial Costs-Benefits Analysis

SDRSocial Discount Rate

SIASocial Impact Assessment

SIAASocial Impact Analysts Association

SRASocial Return Assessment

SRHSexual and Reproductive Health

SROISocial Return on Investment

SSQSix Simple Questions

STDsSexually Transmitted Infections

SVAStakeholder Value Added

TBLTriple Bottom Line

ToCTheory of Change

TSOThird Sector Organisation

UKUnited Kingdom

UNUnited Nations

UNICEFUnited Nations Children’ Fund

VfMValue for Money

WBWorld Bank

WHOWorld Health Organization

1


Introduction

1.1


Purpose of the chapter

This chapter introduces the social return on investment methodology – its definition, history as well as strengths and weaknesses of its application. It then proceeds to identify examples of previous applications of the method, explores any previous intersection of the methodology with the maternal and newborn health field and provides the rationale for the study. Finally, the objectives of this research and an overview of the thesis are provided.

1.2


What is Social Return on Investment?

The first definition of Social Return on Investment (SROI) is “

a simple financial assessment of socio-economic value. SROI compares a project’s net benefits to the investment required to generate those benefits over a certain period of time

” (Emerson and Cabaj, 2000).

Over time, this definition has been modified. In the most recent SROI methodology guidance, SROI is defined as “

a framework for measuring and accounting for the much broader concept of value. It seeks to reduce inequality and environmental degradation and improve wellbeing by incorporating social, environmental and economic costs and benefits

” (Nicholls et al., 2012). This definition is most commonly referred to in the literature, when authors define SROI, probably because it is stated within the SROI guidance.

New Economics Foundation (nef) also defines SROI as “

an analytic tool for measuring and accounting for a much broader concept of value, taking into account social, economic and environmental factors

” (nef, 2014).

SROI measures change from the perspective of stakeholders that experience or contribute to a particular activity, intervention, project, programme or policy. “It tells the story of how change is being created by measuring social, environmental and economic outcomes and uses monetary values to represent them” (Nicholls et al., 2012). Data collection and subsequent analyses allow calculation of a benefits-to-costs ratio. For example, a ratio of 4:1 indicates that an investment of £1 delivers £4 of social value.

There are two types of SROI (Nicholls et al., 2012) [

Figure 1-1

]:

• Evaluative SROI: This retrospectively measures outcomes that have already happened.

• Forecast SROI: This estimates how much social impact will be generated if the activities to be conducted meet their intended outcomes.

Figure 1.1: Types of Social Return on Investment Studies

1.3


History of SROI

A review of the development of the SROI methodology would aid understanding of the initial considerations of those who developed the tool and the capabilities and potentials of the tool. This understanding is critical for this research as it provides a foundation upon which the application of the SROI framework in the maternal and newborn health (MNH) area can be better understood.

Roberts Foundation (RF) developed the initial SROI framework in 1996, presented in a report titled

New Social Entrepreneurs: The Success, Challenge and Lessons of Non-profit Enterprise Creation

(REDF, 1996). The purpose of the first SROI report by REDF was to demonstrate the blend of financial, social and environmental value that all the social enterprises within REDF’s funding portfolio were producing compared to the total investment the organisation was making. This first SROI did not attempt to account for all benefits accrued from a programme, but estimated the cost savings or revenue contributions that are attributed to the programme. The framework utilised a modified discounted cash flow analysis for this calculation to demonstrate impact (Emerson and Cabaj, 2000; Emerson et al., 2000).

In 1997, the RF, under its new initiative – Roberts Foundation Enterprise Development Fund (REDF) updated the framework so to have the capacity to account for total organisational social return on investment and adjusted to be able to account for actual performance that could be continuously updated. REDF recognised more limitations in the methodology (Emerson et al., 2000), but has since not produced any further SROI reports or guidelines. However, the concepts that underpinned the development of this initial framework have been built upon in the subsequent development of the methodology over the past two decades (Tuan, 2008). Guidelines for SROI application have been produced in year 2000 (Emerson and Cabaj, 2000), 2004 (Lingane and Olsen, 2004) and 2006 (Scholten et al., 2006). In 2008, the United Kingdom (UK) Office of the Third Sector (Now referred to as Office for Civil Society (Civil Society Media, 2010)), which is responsible for charities, social enterprises and voluntary organisations in the Cabinet Office (UK Government, 2014) launched the Measuring Social Value project (Arvidson et al., 2010) and this led to the development of an updated guideline in 2009 (Nicholls et al., 2009) and the production of another revision three years later (Nicholls et al., 2012).

These revisions have over the years integrated the initial REDF SROI methodology, which was essentially a social impact measurement tool with principles and processes normally used in economic evaluations and financial return on investment to build a framework that captures social, economic and environmental impacts of interventions (Rotheroe and Richards, 2007). This concept of capturing the broader impacts is widely referred to as the “triple bottom line” (Norman and MacDonald, 2004), which is in itself encapsulated within the “blended value accounting” theory (Emerson, 2003). Furthermore, through this evolution, a more detailed stakeholder analysis is now included, shorter time frames are used for estimations and a process to adjusts the results for outcomes that may be attributable to different organisations has been incorporated into the calculation of the SROI ratio (Tuan, 2008).

Discussions on how best to structure the methodology are on-going and networks such as the European SROI Network (ESROIN), formed in 2004, the SROI Network (international), formed in 2008 and subsequently affiliated networks formed in Sweden, Canada and Australia, are constantly leading and shaping this process (SROIN, 2014a). The methodology is gradually gaining interest in Africa and Asia too. The drive to develop the methodology has come from within and without the third sector, as third sector organisations (TSOs), such as not for profit organisations, charities and voluntary organisations, are increasingly getting involved in delivering public services and are viewed as development partners who need to show ‘value for money’ (Netten et al., 2010). On one hand, TSOs are pushing to become better in being accountable and demonstrating the value of their activities to donors. On the other hand, the focus of funding organisations has shifted from “output” to “outcome” and “impact” and from “generous giving” to “accountable giving”, through which they can demonstrate that evidence-based decisions have informed their choice of funded interventions, projects or programmes (Leat, 2006).

1.4


Application of the SROI methodology

The application of the SROI framework can vary from measurement of the impact and social value generated by a project to a programme or indeed to an entire organisation (Millar and Hall, 2013).

A rapid search of academic databases does not retrieve substantial amounts of research work that have used the SROI methodology, however the methodology is very popular within the third sector and has been widely embraced by the Office of the Civil Society, donors, commissioners and the public sector (Ainsworth, 2010; Heady, 2010). Its wider practice-based use but limited application within academia restricts its development and could potentially be the reason for its minimal uptake (Arvidson et al., 2010; Ebrahim and Rangan, 2010).

Since its first use by REDF in 1996 (Emerson and Cabaj, 2000), the SROI methodology has been broadly applied to many organisations and within many industries. Examples of its use across different sectors include agriculture, where it was used to account for impact of care farming (Leck, 2013); environmental, where it was used to account for impact of a natural regeneration project (Weston and Hong, 2012); energy, where it was used to account for impact of energy efficiency of homes in Germany (Kuckshinrichs et al., 2010); social, where it was used to account for impact of providing training and employment for blind people (Sital-Singh, 2011), transport, where it was used to account for impact of a co-ordinated shared transport service which provides access to workplace, training and childcare in rural Scotland, where no alternative transport is available. (Wright et al., 2009) and indeed health, where amongst other examples, has been used to account for impact of a community-based paediatric asthma programme and provision of highly nutritious breakfast for school children in Sydney, Australia (Bhaumik et al., 2013; Varua and Stenberg, 2009).

As regards its application in health [

described in detail in


Chapter 2

:

systematic review of SROI application in health

], It is worth noting that in 2009, the UK Department of Health (DoH), with the support of the Cabinet Office, commissioned an action research project on which five health focused social enterprises were supported to conduct SROI analysis (Department of Health, 2010). This research demonstrated the relevance of the SROI methodology to the health sector and proved that there were additional benefits of using the methodology (Millar and Hall, 2013).

However, to the best of our knowledge, though relevant and appropriate, there is no reported evidence of the application of the SROI methodology in the field of maternal and newborn health.

1.5


Difference between SROI and other related methodologies

As mentioned above, the SROI methodology has evolved from a combination of social impact assessment tools and economic evaluation tools. It is therefore important to understand how SROI differs from these existing tools and identify what innovative approach it actually offers.

When compared to other social impact measurement tools, there are subtle differences. There are several tools that have been previously or are currently being used in measuring and accounting for social impact (Acumen Fund, 2007; Brest and Harvey, 2008; Brest et al., 2009; Centre for High Impact Philanthropy, 2008; Clark et al., 2004; Emerson et al., 2000; Maas and Liket, 2011; Maas, 2008; Maughan, 2012; Nicholls et al., 2012; Owen and Swift, 2001; Schaltegger et al., 2004).

Table 1‑1: Summary table of social impact assessment methods

Method						Primary application to Date
Process	Outcome	Impact	Monetisation	Non Profit	For Profit	Government
Acumen Scorecard	ï‚Ÿ	ï‚Ÿ			ï‚Ÿ	ï‚Ÿ
Atkinsson Compass Assessment for Investors (ACAFI)	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ			ï‚Ÿ
Balanced Scorecard (BSc)		ï‚Ÿ	ï‚Ÿ		ï‚Ÿ
Benefit-Cost Analysis		ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ		ï‚Ÿ
Best Available Charitable Option (BACO)		ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ
Base of Pyramid (BoP) Impact Assessment Framework		ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ
Cost per Impact		ï‚Ÿ	ï‚Ÿ		ï‚Ÿ		ï‚Ÿ
Expected Return		ï‚Ÿ
Global Reporting Initiative		ï‚Ÿ	ï‚Ÿ				ï‚Ÿ
Local Economic Multiplier (LEM)		ï‚Ÿ			ï‚Ÿ		ï‚Ÿ
Measuring Impact Framework (MIF)		ï‚Ÿ				ï‚Ÿ
Millennium Development Goal Scan (MDG-Scan)		ï‚Ÿ				ï‚Ÿ
Most Significant Change		ï‚Ÿ			ï‚Ÿ
Ongoing Assessment of Social Impact (OASIS)	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ		ï‚Ÿ
Poverty and Social Impact Analysis		ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ
Social Accounting and Auditing (SAA)		ï‚Ÿ	ï‚Ÿ		ï‚Ÿ
Social Costs-Benefits Analysis (SCBA)		ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ
Social Return Assessment (SRA)	ï‚Ÿ	ï‚Ÿ				ï‚Ÿ
Social Return on Investment (SROI)	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ	ï‚Ÿ
Stakeholder Value Added (SVA)		ï‚Ÿ				ï‚Ÿ
Theories of Change	ï‚Ÿ	ï‚Ÿ			ï‚Ÿ

[

Table 1-1

] shows the key characteristics of these different social impact measurement tools described in the literature, identifying the tools that monitor efficiency and effectiveness of outputs, indicators or variables (process); tools that relate outputs and outcomes and to substantiate incremental outcomes above what would have occurred if the intervention was not implemented (impact) and tools that monetize outcomes and/or impact by transforming them to monetary value (monetization). Finally the table details the sector(s) in which the method has thus far been applied – non-profit (third sector), for profit (commercial) or public sector (government).

Of these tools, SROI appears to have gained wide acclaim because of its involvement of all stakeholders, who independently define the value they have experienced because of the intervention. In addition, SROI monetises costs and benefits and also ticks all the boxes of the logical framework, as it accounts for processes, outcomes and impact (Arvidson et al., 2010; Nicholls et al., 2012).

The other similar methodologies to SROI are Economic Evaluation (EE) tools. These compare two or more alternative interventions in terms of their costs and benefits (Drummond et al., 1997). Economists have distinguished four types of economic evaluation: Cost-minimization analysis (CMA), Cost-effectiveness analysis (CEA), Cost benefit analysis (CBA) and Cost-utility analysis (CUA). While the definition of cost remains the same, the manner of accounting for the benefits differs across the types (Drummond et al., 2005). These different types of EE are described below:

• Cost-minimization analysis (CMA): In which, there is proven evidence that equivalent effectiveness of the comparators in question. Thus the more cost-effective option would be the cheaper one (since both options have similar outcome).

• Cost-effectiveness analysis (CEA): Effectiveness is measured in ‘natural units’ such as life years gained.

• Cost benefit analysis (CBA): Costs and benefits are monetised, with the cost-effective option being the one that results in a lower monetised ratio.

• Cost-utility analysis (CUA): Outcome is measured in Quality-adjusted life years (QALYs) or Disability-adjusted life years (DALYs), which is a composite metric of both length and quality of life.

When compared to the SROI methodology, whereas a large focus of economic evaluation tools is on the microeconomic evaluation of the impact of any intervention, SROI accounts for not just economic value, but social and environmental impacts. It has been described as an extension of the cost-benefit analysis that incorporates broader socio-economic outcomes (Westall, 2011). In addition, SROI focuses on stakeholders, uses financial proxies, is more of a management tool, is more likely conducted by organisations themselves, while CBAs are usually conducted by external agents and the most up to date guidance does not recommend comparing estimated SROI ratios, unlike CBAs that are designed to be comparable. However, these differences appear to be largely based on “the style of each approach, rather than the true substance” (Arvidson et al., 2010).

1.6


Strengths and limitations of the SROI methodology

1.6.1


Strengths of the SROI methodology

Practitioners and previous authors have identified some of the key strengths of the SROI methodology.

• Firstly, the fact that the SROI process generates a singular ratio that captures the broader impact (positive and negative) of an intervention is perceived as a key strength (Millar and Hall, 2013).

• One other strength is its meaningful engagement of stakeholders and its representation of their benefits in ways that are unique to the stakeholders themselves (Millar and Hall, 2013).

• More so, the fact that this benefits experience or impact experience is being explained from the perspective of the stakeholder is a unique characteristic of SROI, and this important stakeholder view is often neglected in conventional impact evaluation tools (Rauscher et al., 2012).

• In addition, the method provides a platform for social enterprises to leverage and/or effectively communicate their impact. This ultimately promotes better communication amongst all partners, whether they are beneficiaries, funders or implementers (Mdee et al., 2008).

• The SROI process also strengthens accountability and transparency systems of social enterprises (Arvidson et al., 2010).

• Finally, the SROI process is a very useful auditing tool to aid management decisions and service improvement (Social Ventures Australia (SVA) Consulting, 2012).

1.6.2


Limitations of the SROI methodology

However, there are limitations of the methodology, just as there are of other impact evaluation methodologies (Arvidson et al., 2010).

• Some methodological limitations that have been raised include the difficulty of attaching financial values to “soft outcomes” that require subjective evaluation to generate “financial proxies” (Lingane and Olsen, 2004) such as self-esteem or confidence (Bertotti et al., 2011).

Albeit still in developmental stage, is the Wiki Values, Outcomes and Indicators for Stakeholders (VOIS) database of the SROI Network is an attempt are to standardise such outcomes by recommending indicators and values of such outcomes that can be used to account for them (Rauscher et al., 2012). An example of a health related outcome on WikiVOIS is “change in incidence of abortion”. Contributors to the database have suggested that “cost of medical termination of pregnancy” can be used to value this outcome in Canada (The Global Value Exchange, 2013).

• In addition, the methodology needs an estimation of “what would have happened anyway” to calculate the value that can be attributed to the specific organisation, programme or intervention, however, this counterfactual data is seldom available (Cordery, 2013; Heady, 2010).
• Furthermore, the lack of comparability of SROI ratios across different organisations and/or interventions is perceived as a limitation (Ryan and Lyne, 2008).

• On practical grounds, SROI methodology is adjudged to be cost-intensive and requires the expertise and know-how of professionals and in some cases training, some of which small organisations cannot afford (Wood and Leighton, 2010).

1.7


Rationale for the study

Though applicable and relevant, to the best of our knowledge, there is no documented application of the SROI methodology in the area of maternal and newborn health. For most MNH interventions, the key beneficiaries are the women and their families. There is currently no tool that captures the perspectives of all these stakeholders on impact of any intervention in one summary figure. This underscores the need to explore the use of primary data that will include inputs of those benefiting from MNH interventions, as this may potentially be useful in identifying the most complete estimate of the impact of these interventions. To buttress this, some authors have pointed out that this complete evaluation of MNH interventions needs to include identification of the most culturally appropriate intervention for each local setting and peculiar issues unique to the setting. Those who are the major beneficiaries of the intervention are better positioned to identify these culturally appropriate interventions. Thus the call for culturally sensitive impact evaluations, which attempt to identify specific causes of problems, rather than basing decisions on “superficial” indicators such as maternal mortality rate, which is only quantitative and does not capture softer outcomes that also demonstrate impact of an intervention (McPake and Koblinsky, 2009; Pradhan, 2008).

Secondly, data used as maternal health indicators are usually generated from relatively credible sources such as the Demographics and Health Survey (DHS), Multiple Indicator Cluster Survey (MICS) and World Development Indicators, but the problem of lack of uniformity in the process by which the indicators were generated remains (Pradhan, 2008). This is complicated by the fact that a large number of developing countries, that receive aid, do not have accurate or complete secondary data to base impact evaluation on (WHO,