Jodie Thorpe1
Market systems development (MSD) programmes aim to influence private actor behaviour to promote markets that work better for the poor. This article aims to inform theory-based evaluation (TBE) of such programmes, arguing that a stronger analysis of market actor behaviour change is needed. It proposes a ‘behaviour change framework’ (BCF), building on recent advances in the TBE literature. These focus attention on behaviour change as contingent on the alignment of actor capability, motivation, and opportunity, influenced by the meso and macro contexts. The article applies the BCF to three theory-based MSD evaluations to illustrate its applicability and draw lessons on its use. The BCF can be used to identify evidence gaps and support more compelling explanations of what worked and under what conditions. Such evidence can inform future MSD programmes, and enable them to better stimulate systemic change in line with poverty reduction.
Market system development, theory-based evaluation, behaviour change, motivation, capability, opportunity.
Growing interest in business as a development actor has led donor agencies, governments, non-governmental organisations (NGOs), and businesses to implement support programmes that promote private investment in economic activities that contribute to development goals (Humphrey et al. 2014). One such approach is known as ‘market systems development’ (MSD). MSD aims to systematically understand and intervene in market systems, in order to:
identify the underlying causes (rather than symptoms) of weak market system performance in order to realise large‑scale change. Intervention should continually strive to leverage the actions of key market players to bring about extensive and deep-seated system change.
Sustainability is a prime concern of market systems development. This means considering not just the existing alignment of key market functions and players but how they can work more effectively and inclusively in the future, based on the incentives and capacities of market players.
The approach focuses on stimulating a change in behaviour of market players – public and private, formal and informal – so that they are better able and motivated to perform important market functions effectively.
(Springfield Centre 2014: 3, emphasis added)
MSD interventions seek to influence the behaviours of market actors such that they are better aligned with responsible or inclusive business models, catalysing systemic changes towards more inclusive economies and poverty reduction (see Figure 1). However, this impact chain is highly stylised. The complexity of market systems means interventions do not in fact progress via a fixed or linear plan and also depend deeply on context.
Figure 1 Strategic framework for market systems development programmes
Source: Springfield Centre (2014: 5), reproduced with permission.
Evaluating MSD programmes has proven challenging (ICAI 2014; Creevey et al. 2010; Coffey International Development and M4P Hub 2012; Taylor 2013). As a result, there is a lack of robust evidence and analysis showing how development programmes may best stimulate systemic change in value chains and markets (Campbell 2013; Creevey, Dunn and Farmer 2011). In light of this challenge, theory-based evaluation (TBE) has been recommended for MSD programmes (Jenal and Liesner 2017; O’Sullivan 2016; White 2009). In TBE, evaluators are encouraged to elicit and test different causal chains to understand how outcomes are achieved. Critical elements of high-quality TBE include deep questioning of multiple sources of evidence and an emphasis on why and how processes being evaluated work or not, including assessing underlying assumptions and the contextual factors that influence these processes (Delahais and Toulemonde 2012; Mayne 2008, 2012; Patton 2012; Ton, Vellema and de Ruyter de Wildt 2011; White 2009).
This article argues that evaluators of MSD programmes would benefit from conceptual frameworks that make it easier to identify and assess market actor behaviour change and its relationship to the meso and macro environments. The next section examines how behaviour change is currently discussed in the TBE literature. It concludes with the presentation of a prototype ‘behaviour change framework’ (BCF). Section 3 describes the article’s methodology for applying this framework to assemble and assess evidence from three existing MSD evaluations. Section 4 presents the findings from this process, leading to a discussion of the potential for the BCF to support future MSD evaluations in Section 5. The article’s conclusions follow in Section 6.
This section discusses key concepts relevant to understanding behaviour change in MSD programmes, drawing from literature on TBE. It focuses on the two most mentioned types of TBE: theory of change approaches and realist evaluation (CEE 2012), discussing each in turn.
Theory of change approaches are based on understanding how programme interventions are intended to function, linking activities to outputs, immediate and intermediate outcomes, and impacts, including the assumptions inherent in these causal chains. Mayne (2015) introduces what he describes as more ‘intuitive’ labels to be used in these chains. He uses behaviour change instead of immediate outcomes, direct benefits for longer-term outcomes, and wellbeing changes for impacts. In between outputs and behaviour changes, Mayne also introduces two steps: (1) reach and reaction, and (2) capacity change. Reach and reaction refer to the spread of ideas or incentives to groups targeted by an intervention, and their initial response. In MSD, these groups would be market actors, such as manufacturers, banks, or business service providers. Capacity changes relate to knowledge, attitudes, skills, aspirations, and opportunities (Mayne and Johnson 2015), and are a prerequisite for new actions to be taken.
Further work by Mayne (2018) draws on the COM-B model, a behaviour change system set out in Michie, van Stralen and West (2011), which was developed from a systematic review of behavioural approaches in the health sector. In this system, three elements interact to generate capacity change: (1) motivation, or the internal processes which direct behaviour, including both reflective or analytical processes and more automatic or instinctive habits, norms, and emotional responses; (2) capability, including the physical and psychological capacity to act; and (3) opportunity, or the external factors (outside the individual) that enable or block behaviours, related to the physical, social, or cultural environment and to systems of rules or incentives, which influence an actor’s expectations of reward or punishment. Among these factors, motivation plays a particular role as it involves the choices and habits that energise and direct behaviour (ibid.). Both capability and opportunity can have an impact on motivation, such as by promoting new ways of thinking.
Notably, all three elements need to be present to drive capacity and behaviour change (Darnton 2008; Mayne 2018). Programmes therefore need to establish which of these element(s) are preventing desired behaviours, and design interventions to address gaps. Feedback loops are also a crucial component, and there is often a feedback loop from new behaviours to the future capacities of actors (Mayne 2015). For example, new knowledge regarding market opportunities that has been generated as a result of product innovation might motivate further innovation, while poor results may deter it.
Realist evaluation tests hypotheses about which programme interventions work, for whom, and under what conditions. The focus is on causal mechanisms that motivate actor behaviour, and particularly whether and how programme interventions stimulate new behaviours (Pawson and Tilley 1997; Ton et al. 2011; Jenal and Liesner 2017). Recognising that these processes are contingent on context, the hypotheses to be tested are expressed in the form of Context-Mechanism-Outcome (CMO) configurations.
In relation to MSD, we can define context to include institutional, organisational, socioeconomic, and cultural conditions and resources affecting specific (groups of) market actors. Outcomes are observable behavioural changes stemming from these actors’ decisions, which are influenced by context and by programme interventions. Mechanisms are key to behaviour change. They are the incentive structures that shape actor decisions, and which programme interventions aim to influence. Realist evaluation also recognises feedback loops through which outcomes may influence (strengthen or dampen) causal mechanisms.
There is debate in the literature over the nature of causal mechanisms. While Pawson and Tilley (1997) have explained mechanisms as being related to actor reasoning and resources, others (Westhorp 2018; Ton 2021) define them as working at different levels of social systems. Mechanisms therefore include ‘the inner motivations of people and firms’ as well as ‘the power of structures that shape or constrain their agency’ (Ton 2021).
Westhorp argues that multiple constructs are needed in order to assess how and why programme interventions work, particularly when viewed from a whole system perspective.
Here is the crux of the issue: the causal properties of systems are not solely reducible to the decision-making of people within those systems. The implication for evaluation is equally clear. If programmes are indeed social systems, as Pawson and Tilley have eloquently argued, then the causal properties of the programmes are by definition not reducible solely to the decision-making of the targeted individuals.
(Westhorp 2018: 8)
Instead, Westhorp suggests that mechanisms operate across different system levels, which may include material (biochemical, physical), individual, social-group, and social-institutional. Her key contribution is to emphasise that systemic change happens across these levels, sometimes in different time frames.
Realism has long acknowledged that mechanisms operate at different levels of the system than their outcomes… It is necessary to look to the sub-systems – of what they are comprised, what they do and how they do it, and what the consequences of their operations are – in order to understand how a system – or some aspect of it – works. However, realism also acknowledges that causation works downwards, as well as upwards.
(ibid.: 5).
Based on ideas drawn from both theory of change and realist perspectives, Figure 2 presents a framework for evaluating behaviour change in MSD programmes. This behaviour change framework (BCF) integrates the COM-B model (Michie et al. 2011), as presented in Mayne (2018), and Westhorp’s (2018) insight that mechanisms operate at different system levels. It has resonances with the framework presented in the introduction to this edition, as well as other work on stakeholder behaviour in value chains (Ton et al. 2021; Ton 2021).
Figure 2 Behaviour change framework for market systems
Source Author’s own, based on concepts from Mayne (2015, 2018), Michie et al. (2011), Westhorp (2018). See also Ton et al. (2021) and Ton (2021).
At the centre is a market actor, such as a firm or enterprise, that the programme seeks to reach and influence in order to produce behaviours in line with poverty reduction. Actor behaviour is determined by capacity, which is the product of capability + opportunity + motivation. Although the actors in MSD are primarily composite actors like firms, rather than individuals, their behaviours still result from coordinated actions by the individuals involved. Where MSD programmes have multiple components that target different groups of actors, the BCF would be applied separately to each group of interest.
MSD programmes catalyse sustainable changes in market actor capacity by avoiding direct solutions at micro level. Instead, they influence the availability or quality of meso and macro level support functions, services, and institutions which in turn influence actor capacity. In realist terms, these meso and macro interventions create mechanisms that motivate actors towards desired behaviours. The meso level involves interactions among diverse actors (e.g. within value chains, clusters, networks, communities), and may comprise proto-institutions, such as voluntary standards or multi-stakeholder initiatives. The macro level involves structural factors that work across economies or societies, based on formal and informal institutions (van Wijk et al. 2019). The BCF (Figure 2) maps programme pathways from interventions to meso- or macro-level outcomes, and ultimately to actor capacity change at micro level.
In the BCF, all three elements of capability, opportunity, and motivation need to be in place in order to generate the desired behaviours and outcomes. This condition is achieved through a combination of pre-existing contextual factors and programme mechanisms. The programme’s assumptions and intentions regarding these three elements can also be highlighted (see the table ‘Programme mechanisms’ at the bottom of Figure 2).
Embedded in these processes are feedback loops through which outcomes may amplify or dampen their causes, indicated as double-headed arrows in Figure 2. Feedback loops often link market actor behaviour and market actor capacity, for example. They may also link micro, meso, and macro levels of the market system, since micro-level changes in the behaviour of actors can also contribute to new meso and macro contexts (Westhorp 2018).
The rest of this article investigates the applicability and added value of the BCF in TBEs. To do so, it applies the framework to three existing MSD evaluations, identifying, assembling, and re‑examining evidence across macro, meso, and micro levels. Publicly available programme evaluations are chosen for this study as they present a comprehensive account of MSD programme results, reflecting on the systemic nature of outcomes achieved. Such evaluations are frequently used as a core source of learning by programme funders and implementers to understand what works, under what conditions.
The three evaluations were selected from the Building Effective and Accessible Markets (BEAM) Exchange evidence map (BEAM Exchange 2018), a database of published resources that investigate the connection between MSD interventions and programme results. In this map, evaluations are tagged based on the results level that they illustrate. As of 1 April 2021, the database contained 90 sources which presented ‘high confidence’ evidence and learning on MSD effectiveness. Twelve are independent, theory-based impact evaluations, of which three were selected for this study (see Table 1). These three were chosen because they (a) primarily illustrate the ‘intervention’ or ‘systemic change’ results levels, which were expected to provide a deeper and richer discussion of market actor behaviour change, and (b) represent a variety of contexts, covering the education, finance, and agriculture sectors across three countries in Africa and Asia.
Table 1 Sample of theory-based MSD evaluations analysed
| Programme | Programme aim | Source | Type of evaluation | Evaluation focus | Target market actor assessed |
|---|---|---|---|---|---|
| Developing Effective Private Education Nigeria (DEEPEN) | Improve the quality of education provided by private schools in Lagos | MacAuslan et al. (2018) | Theory of change | Whole programme | Private schools in Lagos |
| Financial Sector Deepening Trust Kenya (FSDK) | Generate sustainable livelihood improvements through better financial sector capacity and operations | Stone, Johnson and Hayes (2010) | Theory of change | Sample of 13 projects | Equity Bank |
| (5 micro, 4 meso, 4 macro) | |||||
| Oxfam’s Gender Transformative and Responsible Agribusiness Investments in South‑East Asia (GRAISEA) | Improve livelihoods of women and men small‑scale producers through more responsible and inclusive value chains and private sector investments | Tobing-David (2019) | Realist | Whole programme | Vietnamese agribusiness |
Source: Author's own.
The application of the BCF to these evaluations involved three steps:
Gender Transformative and Responsible Agribusiness Investments in South-East Asia (GRAISEA) was an Oxfam programme that aimed to improve the livelihoods of small-scale producers through catalysing more responsible and inclusive private sector activity. It targeted leading agribusinesses, financial institutions, multi-stakeholder initiatives, and national legislation in support of more sustainable production practices in four value chains across seven Asian countries. The evaluation analysed the programme’s contributions in four results areas, identifying and discussing the most significant outcomes that ‘theoretically showed the strongest logical link and empirically demonstrated positive results’ (Tobing-David 2019: 8). The evaluation explicitly uses a realist approach, exploring strategy effectiveness with respect to seven causal mechanisms.
To illustrate the use of the BCF, this article focuses on what the evaluation terms ‘Result 3’. In this component, GRAISEA sought to catalyse Asian agribusinesses to adopt corporate social responsibility (CSR) policies that support small-scale producers and gender equity. The most significant outcome identified by the evaluation was the adoption of gender-transformative CSR policies and plans in Vietnam, with 31 seafood companies adopting gendered CSR guidelines and reporting, and 13 companies reporting full compliance (ibid.: 26). Figure 3 reproduces a segment of a diagram from the evaluation report which illustrates this outcome trajectory.
Figure 3 GRAISEA’s most significant outcome of ‘result 3’
Note: The darker shade indicates the pathway to the most significant outcome. The original diagram also included trajectories for the Philippines, Myanmar, Thailand, and Cambodia, but these were not labelled as being ‘most significant’. For the sake of simplicity and clarity, these have been left out here. MSIs refers to multi‑stakeholder initiatives.
Source: Author’s own, adapted from Tobing-David (2019: 26).
The evaluation concludes that three critical factors contributed to GRAISEA’s achievements. Firstly,
in a country like Vietnam where there is a strong state presence, the government plays a truly defining role. Secondly, international standards imposed by the export market mean that CSR has a commercial value, and lastly, Oxfam in Vietnam made CSR practices more ‘practical’ by introducing gender CSR Guidelines and Sustainability Index reporting for the companies to experience it.
(ibid.: 38)
Compatibility between companies’ values and the programme’s goals was another contributory factor (ibid.: 36).
In contrast, the evaluation found that programme activities to convene and influence multi-stakeholder initiatives (MSIs) were not impactful. GRAISEA had assumed that large corporates were highly engaged in MSIs, and highly likely to make reference to MSI guidelines in their business strategies. However, there was little evidence that these assumptions held true (ibid.: 38).
Figure 4 presents GRAISEA’s evidence through the BCF lens. There are similarities with GRAISEA’s outcome trajectory, especially in framing pathways in terms of target actors and desired behaviours. However, the BCF introduces an additional pathway at macro level, drawing on detail from other sections of the evaluation. Through the BCF, these macro, meso, and micro interventions are assessed with respect to supporting capability, opportunity, and motivation, adding detail from the evaluation report on programme mechanisms and contextual factors.
Figure 4 Capacity and behaviour change of agribusiness in Vietnam
Source: Author’s summary of factors linked to the adoption of CSR by Vietnamese agribusinesses, based on Tobing-David (2019).
The BCF shows GRAISEA playing a key role in providing access to knowledge and tools, which foster technical capability for CSR within the private sector. It does raise a question, however, highlighted in italics (in Figure 4), of whether this direct delivery approach has generated sector-wide effects, in line with MSD systems thinking.
Comparing the BCF to Figure 3, the BCF adds most detail with respect to opportunity. At macro level, it adds the programme’s work on gender-sensitive CSR guidelines and the government’s adoption of CSR elements in Vietnam’s national shrimp strategy (ibid.: 18). In the context of a strong state, this development is likely to have had an important bearing on companies’ perceptions of opportunities from CSR, either in the form of rewards or punishments. While this pathway is clear in the BCF, it is only represented in Figure 3 through a reference to ‘key influencers’.2
At meso level, the BCF (Figure 4) presents a similar picture to Figure 3, with both indicating weak opportunities resulting from GRAISEA’s work with MSIs. However, the BCF also draws out an important contextual factor showing that standards imposed in export markets influence opportunities for Vietnamese companies trading outside the country.
Finally, although not visible in Figure 3, the evaluation identifies ways in which GRAISEA affected companies’ motivations. This is described as giving ‘space for companies to understand and experience its relative advantage’ from using CSR tools (ibid.: 36), which was further enabled by the trust that Oxfam built with these companies. The evaluation also identifies more intrinsic motivations, notably company awareness and commitment to act responsibly, especially in the case of micro, small and medium-sized enterprises (MSMEs). ‘Unless there is a strong drive to adopt responsible business conduct policies and practices, especially ones that recognize the significant role of women, these MSMEs will be less likely to adopt’ (ibid.: 13).
Financial Sector Deepening Trust Kenya (FSDK) aimed to support the development of an inclusive Kenyan financial sector, building on an earlier programme of technical support. Both programmes were funded by the UK’s Department for International Development (DFID).3 FSDK developed a portfolio of 34 projects targeting policy and regulatory change, sector support services, and retail banking capacity. Together, these projects were intended to impact the capacity and operation of the sector, and to generate sustainable livelihood improvement for poor Kenyans. The evaluation focuses on the validity of the FSDK impact pathways across a sample of these projects using a TBE framing. ‘The aim… was, first, to establish the theoretical programme impact pathways and, second, to obtain evidence that can substantiate (or refute) the effective functioning of these pathways in practice’ (Stone, Johnson and Hayes 2010: 6).
The evaluation report discusses changes at three levels, which it explicitly defines as macro (policy and regulation), meso (sector support services), and micro (retail). It finds a significant contribution of FSDK at all three levels (ibid.: v), and highlights strong synergies, citing Equity Bank as a key example (ibid.: 17). FSDK helped Equity, a former building society, to transform into a bank, while its support for MicroSave, a consulting company providing product development support, contributed to the Equity Bank’s subsequent expansion. Policy influencing, enabled by DFID, also contributed to this transformation and growth.
Figure 5 presents evaluation evidence through the BCF lens, using Equity Bank as the example. While the FSDK evaluation contains a very similar visual of micro, meso, and macro levels of support in a nested structure (ibid.: 5), it does not extend the use of this visual to present specific outcome pathways, nor does it relate interventions to changes in capability, opportunity, or motivation. The BCF fills these gaps, and in doing so, helps to illustrate and explain synergies across FSDK pathways.
Figure 5 Capacity and behaviour change of Equity Bank
Source: Author’s summary of factors linked to changes at Equity Bank, based on Stone et al. (2010).
Early macro-level interventions supported by DFID4 played a role in influencing Kenyan policymakers, smoothing the way for Equity’s evolution into a bank, and enabling new growth opportunities. Interventions at both micro and meso levels supported new capabilities. Micro interventions provided technical assistance for Equity’s upgrading. Meso-level interventions targeted MicroSave, enabling it to develop the financial solutions that would inform new product development at Equity Bank. The evaluation presents no specific evidence relevant to motivations, which may imply that capability and opportunity together motivate innovation.
On the other hand, the evaluation does highlight micro to meso links, indicated by the double-ended black arrow in Figure 5. It finds that Equity Bank’s successes have impacted on the culture of the wider finance sector, by demonstrating market opportunities for different customer segments (ibid.: 9). As a result, mainstream banks are beginning to compete in lower-income markets (ibid.: 12).
Despite these successes, the evaluation found that poorer clientele were still missing out.
Compared with 2006, we found that Equity had clearly more than proportionately increased its outreach to the rural population, women, younger people and the less-educated… [but] it has not clearly achieved outreach to a poorer clientele any more than has the rest of the banking sector.
(ibid.: 11–12)
This weakness is indicated in Figure 5 by the white arrow between Equity Bank’s capacity and serving low-income groups. Unfortunately, it is not clear from the evidence presented whether the barrier lies with capability, opportunity, or motivation, although the evaluation does question the suitability of Equity Bank’s accounts from the perspective of poorer clients.
In future, the BCF could be applied to help evaluators probe such issues more deeply. Is this primarily a capability issue, affecting Equity Bank’s product portfolio, as the evaluation seems to suggest? Or is the root cause at the level of market opportunity? The only causal mechanism linked to opportunity for Equity Bank is rooted in its transformation from a building society, but is it realistic to think that this change created new opportunities that motivated Equity to work more closely with ‘base-of-the-pyramid’ customers? Perhaps other measures such as tax incentives or universal service obligations would be needed? Or perhaps the root cause lies with intrinsic habits and norms that shape the bank’s motivation to serve this sector?
Developing Effective Private Education Nigeria (DEEPEN) aimed to improve private school education in Lagos state. It sought to address core constraints, mainly information asymmetries, especially parents’ information about school quality; missing support functions, including access to finance and teacher training services; and an unreceptive regulatory regime that left many schools operating informally. By facilitating innovations, DEEPEN intended to improve the quality of education delivered by private schools, particularly those serving poor children. The evaluation ‘assesses DEEPEN by following its theory of change and gathering data on the key assumptions and context, as well as expected outputs and outcomes’ (MacAuslan et al. 2018: 4). It gathered evidence across the programme’s four workstreams: (a) rules and standards, (b) information, (c) finance, and (d) school improvement.
The evaluation found that with the exception of the information workstream, DEEPEN made considerable progress. It influenced government perceptions, policies, and practices towards private schools, and supported credit provider Accion Bank to develop a low-cost financial product. Service providers also developed new and affordable school improvement programmes, although these were out of reach of the poorest schools (ibid.: iv). However, the evaluation also identified major limitations to DEEPEN’s outcomes. There were ‘only very modest changes in behaviour in the low‑cost schools that were surveyed’ (ibid.: 10), in terms of improved capacity or better learning conditions. These schools struggled to access credit or pay for improvement programmes, despite increased affordability. Overall, few gains were detected in pupils’ learning outcomes.
Failure of the state government to fully implement the Graded Assessment of Private Schools (GAPS) legislation was identified as a major factor (ibid.: v). GAPS had been intended to rate the quality of private schools, with results made available to parents and the media. DEEPEN expected that these stakeholders would then generate incentives for schools to invest in quality improvement. However, a change of government in 2015 unexpectedly restricted the roll-out of GAPS. According to the evaluation, this severely constrained the impact potential of all of DEEPEN’s interventions (ibid.: 31).
Figure 6 presents the evaluation evidence viewed through the BCF lens. It provides a new visual representation of the impact pathways and their interactions as described in the evaluation. It also encourages thinking about how these pathways affect motivation, capability, and opportunity, rather than focusing too narrowly on the GAPS policy failure. The elements highlighted in white indicate multiple breakdowns in mechanisms affecting opportunity, which together explain the modest results achieved by DEEPEN.
Figure 6 Capacity and behaviour change of private schools in Lagos
Source: Author’s summary of factors, based on MacAuslan et al. (2018).
On the positive side, DEEPEN’s efforts to strengthen schools’ capabilities through engaging service and credit providers broadly functioned well, although very low-cost schools remained unable to access credit or afford improvement programmes. With respect to opportunity, however, DEEPEN had intended to influence both macro and meso environments in ways that would generate rewards for schools investing in improvements. Although DEEPEN did succeed in improving the state government’s recognition and support for private schools, positively influencing their external environment, other key market drivers were missing.
As the evaluation identified, the problems started with the failure of the state government to fully implement GAPS, which would have delivered important information on school quality. However, the weaknesses in the intended impact pathways do not stop there. Even when DEEPEN attempted to compensate for GAPS failings by directly giving media outlets technical assistance and financial support for educational programming, there is little evidence that parents were actually listening to the radio for information on education and school quality (ibid.: 16). Hence the intended causal mechanism involving parents identifying and paying for higher quality schools is weak. In addition, the findings suggested that the educational programming supported by DEEPEN is unsustainable beyond the end of the programme, as it does not align with the commercial interests of the radio stations (ibid.: 16), which will not pursue it.
As for FSDK, there is no discussion of causal mechanisms or contextual factors linked to motivations, suggesting implicitly that capability and opportunity together would be sufficient. In addition, DEEPEN also assumed that competition would be an important causal mechanism to scale up change. However, in contrast to FSDK, competition did not play this expected role and this link is highlighted in white in Figure 6. In part, this finding reflects the weak opportunities already discussed. However, the evaluation also finds that:
while competition plays a role, and some of the proprietors who were interviewed by the endline evaluation team indeed felt protective of their know-how, there appears to be a high degree of collaboration. This is consistent with the finding that private schools do not always operate on market logic, and that many see themselves more as social enterprises or charitable organisations that are serving an important need.
(ibid.: 28)
Finally, the BCF also draws attention to a potentially important negative feedback loop through which the outcome (improved school quality) limits or undermines capacity for change, particularly in low-cost schools. In these settings, studies have shown that there is frequent teacher turnover, leaving school proprietors reluctant to invest in teacher training (ibid.: 28). The reasons for the turnover are unclear and are likely to be complex. However, to the degree that training enables teachers to access better jobs elsewhere, it would represent a negative feedback loop.
The findings demonstrate how the BCF can enrich TBEs, by aiding evaluators to visually represent and systematically assess market actor behaviour change. In the case of DEEPEN, the framework helps to focus the findings on the (lack of) opportunities for private schools to improve. With FSDK, it encourages deeper understanding of the intersection of micro, meso, and macro factors as they relate to Equity Bank’s capacity. Within the scope of this article, the result is a stronger visual and narrative of Equity’s successes. However, the BCF could also be used to seek new evidence to better understand the programme’s failures to benefit lower-income groups. In the case of GRAISEA, the BCF brings together programme mechanisms and contextual factors currently discussed across the report and shows how they interact to contribute to CSR adoption in Vietnam.
Based on these findings, this section draws out lessons for evaluators – and by extension for those who use the results of TBEs. These insights relate to the drivers of behaviour change for different private sector actors, the interactions between these drivers at micro, meso and macro levels, and the use of the BCF to capture these dynamics.
Fundamentally, the BCF provides a framework for those conducting TBEs to bring together and think critically about multiple sources of evidence relevant to assessing market actor behaviour change. The key is recognising that capability, opportunity, and motivation must all be present and aligned with the desired behavioural outcomes (Darnton 2008; Mayne 2018). These factors may either be pre-existing or be catalysed through programme interventions. Capability and opportunity together influence motivation, although programme interventions may also directly contribute.
Across all three evaluations, programmes were generally successful in their technical and financial support to build company capabilities. However, the findings suggest that evaluators should be particularly interested in assessing opportunity for change. For both DEEPEN and GRAISEA, impact pathways targeting opportunity were hampered by problematic assumptions which the BCF helped to highlight. Understanding motivations can also help to explain how and why change happens. While motivations may be difficult to observe directly, they can be probed with respect to decision-making or preferences.
In this respect, the BCF can be used to enrich what has been termed the ‘will-skill’ framework within MSD practice (Springfield Centre 2014). According to this framework, MSD interventions may address market actor capability (‘skill’) or their incentives and motivations (‘will’). Where capabilities are high but motivations are weak, programmes can focus on making the case for change, for example, or on reducing companies’ perceptions of risk through co-funding investments. However, where actors already have high will and high skill but are not exhibiting the desired behaviour, it implies that obstacles lie in the external landscape (i.e. related to opportunity). The BCF thus offers a will-skill-opportunity framework, and can help programmes and evaluators think more about the meso and macro factors shaping opportunity.
Finally, the BCF encourages evaluators to pay greater attention to feedback loops and particularly the ways in which behaviour change outcomes influence actor capacity. Such dynamics were not explicitly discussed in any of the three evaluations included in this article, despite their prevalence in systemic change processes. However, in the case of DEEPEN, a potential negative feedback loop was identified in which teacher training that was intended to raise the capacity of low-income schools may lead to teachers using their new skills to seek better opportunities elsewhere, returning the school to its low-capacity state. The BCF can prompt evaluators to ask more questions about such loops.
Studies applying the COM-B system have mostly been concerned with health interventions to change individual behaviours in areas such as smoking (Barker, Atkins and de Lusignan 2016; Gould et al. 2017; Suntornsut et al. 2016). However, the BCF extends the use of COM-B concepts to the composite market actors that are the focus of MSD programming. For composite actors, choices, decisions, and behaviours reflect ‘the joint intended effect of coordinated action as expected by the participating individuals’ (Scharpf 1997: 52). These composite actors are influenced not only by an objective (‘rational’) analysis of self-interest but also by subjective motivations. For example, for GRAISEA in Vietnam, the compatibility of MSMEs’ values with CSR activities was identified as an important factor in companies’ capacity to adopt CSR.
Given the complexity of MSD programmes, one challenge can be to identify which actor(s) should be the focus of the micro‑level behaviour change in the BCF. In the case of GRAISEA, for example, interventions targeted policymakers, MSIs, and agribusinesses. However, the central actor in the BCF is the one whose behaviour is directly affecting poverty outcomes, and whose incentives MSD programmes seek to change. For GRAISEA, these are Asian agribusinesses. That said, the meso and macro layers are also populated by actors, who could theoretically be analysed using the BCF lens. In the case of GRAISEA, Oxfam built trust with and offered expertise to policymakers, enabling new behaviours, in the form of CSR policy decisions. The designation of micro vs meso and macro within the BCF is fundamentally an analytical choice, shaped by a programme’s theory of change and the evaluator’s questions.
The BCF can also be used to think in a more granular way about actor behaviour change and particularly the motivations of different target actors. Taking the case of DEEPEN again, the outcomes for schools that serve the poorest children were found to be much more modest than for the others. These schools’ motivations were affected by lower financial and technical capabilities, including higher teacher turnover, and they are also likely to face different opportunities than more affluent schools. Another finding from the DEEPEN evaluation is that the motivations of schools that act as social or community enterprises are different from fully commercial providers. Linking this finding to the BCF suggests that collaboration rather than, or in addition to, competition can be an important mechanism to support the scaling of micro-level behaviour changes to the wider sector. Finally, for FSDK, the BCF helped to highlight the evaluation’s finding that Equity Bank’s ability to serve low-income groups was limited. It could also be used to assess the root causes of this constraint, whether linked to opportunity, capability, or motivation.
Alongside focusing on composite actors, the BCF extends the use of COM-B ideas to systematically capture behaviour change drivers at meso and macro levels. In the case of FSDK, the evaluation had already discussed changes in macro, meso, and micro terms. However, GRAISEA and DEEPEN did not use these designations and Figure 4 and Figure 6 show how their results can be mapped in this way. Admittedly, the more linear outcome trajectories presented in the GRAISEA evaluation (Figure 3) are simpler to understand. However, the price of this simplicity is that many important elements which Figure 4 readily captures are buried in long passages of text.
The BCF can be used not only to represent how macro- and meso-level interventions shape micro-actor behaviour, but also how new micro-level behaviours influence the meso and macro contexts. Both FSDK and DEEPEN, for example, expected scale to emerge through the demonstration effects from micro-level behaviour interacting with competition at sector level, although this mechanism was more effective in the case of FSDK than DEEPEN. These dynamics are represented in the BCF through the arrows linking micro to meso levels.
Using the BCF to lay out and evaluate impact pathways connecting these multiple levels aligns with realist understanding that causal mechanisms of change operate at a different system level than their outcomes (Bhaskar 1997; Westhorp 2018). In this sense, the BCF also aligns with structuration theory (Giddens 1984) and ‘actor-centred institutionalism’ (Mayntz and Scharpf 1995; Scharpf 1997) which emphasise that social phenomena are the product of the interaction between intentional choices by actors, and the institutional context in which they occur.
This article aims to inform the design of theory-based evaluations for market systems development programmes through encouraging a stronger analysis of market actor behaviour change. It develops and tests a new behaviour change framework (Figure 2), which has been informed by ideas discussed in the TBE literature. At the centre is the COM-B model (Michie et al. 2011; Mayne 2018), showing that stimulating particular behaviours requires that capability, opportunity, and motivation are all present. In addition, inspired by Westhorp (2018), the BCF shows how micro-level behaviour change needs to be understood with respect to multiple system levels. Dynamics in the macro and meso environment create the conditions for behaviour change, while micro-level behaviours can condition the meso and macro environment for others.
In the three MSD evaluations discussed in this article, interventions and assumptions related to capability, opportunity, and motivation were readily detected. However, the BCF requirement that all three of these elements align with the desired behaviours encourages deeper critical thinking. In this way, the BCF enables evaluators to seek new evidence and/or more compelling explanations of what has worked within MSD programmes, under what conditions, as well as to explain why programmes miss the mark. Considering the findings presented here, programmes and evaluators could pay more attention to whether and how technical capabilities supported by programmes are matched by meaningful opportunities and aligned with companies’ conscious and intrinsic motivations.
As the BCF is actor-focused, it encourages evaluators to think much more carefully about whose behaviour is being changed, with due attention to nuanced differences in opportunities, capabilities, and motivations. This also means identifying problematic assumptions with respect to the motivations of different types of enterprises or the opportunities available in different market segments. It was notable that in two of the three cases explored in this article, programme successes did not adequately translate to benefits for low-income groups. If future TBEs could generate better understanding of these dynamics, it would provide valuable insights regarding how MSD programmes can better stimulate systemic change in line with poverty reduction.
1 Jodie Thorpe, Research Fellow, Institute of Development Studies, University of Sussex, UK.
2 This very general reference to ‘key influencers’ is likely to have been a simplification to fit the diagram. However, it gives little visibility to the important role of state support, which is described elsewhere in the evaluation as being ‘truly defining’.
3 Now the Foreign, Commonwealth & Development Office (FCDO).
4 FSDK’s support for Equity’s transformation was a continuation of an earlier DFID-funded programme.
Barker, F.; Atkins, L. and de Lusignan, S. (2016) ‘Applying the COM-B Behaviour Model and Behaviour Change Wheel to Develop an Intervention to Improve Hearing-Aid Use in Adult Auditory Rehabilitation’, International Journal of Audiology 55. suppl 3: S90–S98, DOI: 10.3109/14992027.2015.1120894 (accessed 6 October 2021)
BEAM Exchange (2018) Evidence Map (accessed 18 October 2021)
Bhaskar, R. (1997) A Realist Theory of Science, London and New York NY: Verso
Campbell, R. (2013) Feed the Future Learning Agenda Literature Review: Expanded Markets, Value Chains, and Increased Investment, Rockville MD: Westat
CEE (2012) Theory-Based Approaches to Evaluation: Concepts and Practices, Ottawa: Treasury Board of Canada Secretariat Centre of Excellence for Evaluation
Coffey International Development and M4P Hub (2012) M4P Evaluation Workshop Executive Summary, Reading: Coffey International Development
Creevey, L.; Dunn, E. and Farmer, E. (2011) Outreach, Outcomes and Sustainability in Value Chain Projects, microREPORT 171, Washington DC: United States Agency for International Development
Creevey, L.; Dunn, E.; Northrip, Z.; Snodgrass, D. and Cogan Wares, A. (2010) Assessing the Effectiveness of Economic Growth Programs: Private Sector Development Impact Assessment Initiative, Washington DC: United States Agency for International Development
Darnton, A. (2008) Practical Guide: An Overview of Behaviour Change Models and Their Uses, London: Government Social Research Unit, HM Treasury (accessed 10 June 2021)
Delahais, T. and Toulemonde, J. (2012) ‘Applying Contribution Analysis: Lessons from Five Years of Practice’, Evaluation 18.3: 281–93
Giddens, A. (1984) The Constitution of Society: Outline of the Theory of Structuration, Cambridge: Polity Press
Gould, G.S. et al. (2017) ‘Designing an Implementation Intervention with the Behaviour Change Wheel for Health Provider Smoking Cessation Care for Australian Indigenous Pregnant Women’, Implementation Science 12.114, DOI: 10.1186/s13012-017-0645-1
Humphrey, J.; Spratt, S.; Thorpe, J. and Henson, S. (2014) Understanding and Enhancing the Role of Business in International Development: A Conceptual Framework and Agenda for Research, IDS Working Paper 440, Brighton: Institute of Development Studies (accessed 6 October 2021)
ICAI (2014) DFID’s Private Sector Development Work, Report 35, London: Independent Commission for Aid Impact
Jenal, M. and Liesner, M. (2017) Causality and Attribution in Market Systems Development, London: BEAM Exchange
MacAuslan, I.; Sutoris, P.; Findlay, A.; Durieux, M. and Bahri, S. (2018) Developing Effective Private Education Nigeria (DEEPEN) Endline Evaluation Volume 1: Synthesis, Abuja: Edoren (accessed 7 October 2021)
Mayne, J. (2018) ‘The COM-B Theory of Change Model’, Working Paper version 4, unpublished (accessed 6 January 2021)
Mayne, J. (2015) ‘Useful Theory of Change Models’, Canadian Journal of Program Evaluation 30.2: 119–42, DOI: 10.3138/cjpe.230
Mayne, J. (2012) ‘Contribution Analysis: Coming of Age?’, Evaluation 18.3: 270–80, DOI: 10.1177/1356389012451663 (accessed 6 October 2021)
Mayne, J. (2008) ‘Contribution Analysis: An Approach to Exploring Cause and Effect’, ILAC Brief 16, Rome: Institutional Learning and Change Initiative
Mayne, J. and Johnson, N. (2015) ‘Using Theories of Change in the CGIAR Research Program on Agriculture for Nutrition and Health’, Evaluation 21.4: 407–28
Mayntz, R. and Scharpf, F. (1995) ‘Der Ansatz des akteurzentrierten Institutionalismus’, in R. Mayntz and F. Scharpf (eds), Gesellschaftliche Selbstregelung und Politische Steuerung, Frankfurt: Campus
Michie, S.; van Stralen, M.M. and West, R. (2011) ‘The Behaviour Change Wheel: A New Method for Characterising and Designing Behaviour Change Interventions’, Implementation Science 6.42, DOI: 10.1186/1748-5908-6-42 (accessed 6 October 2021)
O’Sullivan, F. (2016) Impact Evaluations for Market Systems Programmes, London: BEAM Exchange
Patton, M.Q. (2012) ‘A Utilization-Focused Approach to Contribution Analysis’, Evaluation 18.3: 364–77
Pawson, R. and Tilley, N. (1997) Realistic Evaluation, London: SAGE
Scharpf, F.W. (1997) Games Real Actors Play: Actor-Centered Institutionalism in Policy Research, Boulder CO: Westview Press
Springfield Centre (2014) The Operational Guide for the Making Markets Work for the Poor (M4P) Approach, 2nd ed., Durham: Springfield Centre
Stone, R.; Johnson, S. and Hayes, J. (2010) FSD Kenya Impact Assessment Summary Report, Nairobi: Financial Sector Deepening Kenya (accessed 7 October 2021)
Suntornsut, P. et al. (2016) ‘Barriers and Recommended Interventions to Prevent Melioidosis in Northeast Thailand: A Focus Group Study Using the Behaviour Change Wheel’, PLoS Neglected Tropical Diseases 10.7: e0004823, DOI: 10.1371/journal.pntd.0004823 (accessed 6 October 2021)
Taylor, B. (2013) Evidence-Based Policy and Systemic Change: Conflicting Trends?, Springfield Working Paper 1, Durham: Springfield Centre
Tobing-David, V.E. (2019) Final Evaluation: Gender Transformative and Responsible Agribusiness Investments in South-East Asia (GRAISEA) Programme. Period: 1 April 2015–30 June 2018, Oxfam Evaluation Report, Oxford: Oxfam International (accessed 7 October 2021)
Ton, G. (2021) ‘Development Policy and Impact Evaluation: Learning and Accountability in Private Sector Development’, in H. Zafarullah and A.S. Huque (eds), Handbook of Development Policy, Cheltenham: Edward Elgar
Ton, G.; Vellema, S. and de Ruyter de Wildt, M. (2011) ‘Development Impacts of Value Chain Interventions: How to Collect Credible Evidence and Draw Valid Conclusions in Impact Evaluations?’, Journal on Chain and Network Science 11.1: 69–84
Ton, G.; Thorpe, J.; Egyir, I. and Szyp, C. (2021) Value Chain Governance: Entrance Points for Interventions to Address Children’s Harmful Work in Agriculture, ACHA Working Paper 6, Brighton: Action on Children’s Harmful Work in African Agriculture, Institute of Development Studies, DOI: 10.19088/ACHA.2021.001 (accessed 18 October 2021)
van Wijk, J.; Zietsma, C.; Dorado, S.; de Bakker, F.G.A. and Martí, I. (2019) ‘Social Innovation: Integrating Micro, Meso, and Macro Level Insights from Institutional Theory’, Business and Society 58.5: 887–918, DOI: 10.1177/0007650318789104
Westhorp, G. (2018) ‘Understanding Mechanisms in Realist Evaluation and Research’, in N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan and S. Dalkin (eds), Doing Realist Research, London: SAGE
White, H. (2009) ‘Theory-Based Impact Evaluation: Principles and Practice’, Journal of Development Effectiveness 1.3: 271–84, DOI: 10.1080/19439340903114628 (accessed 6 October 2021)
© 2022 The Author. IDS Bulletin © Institute of Development Studies | DOI: 10.19088/1968-2022.109
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC BY-NC), which permits use, distribution and reproduction in any medium, provided the original authors and source are credited, any modifications or adaptations are indicated, and the work is not used for commercial purposes.
The IDS Bulletin is published by Institute of Development Studies, Library Road, Brighton BN1 9RE, UK. This article is part of IDS Bulletin Vol. 53 No. 1 February 2022 ‘Theory-Based Evaluation of Inclusive Business Programmes’; the Introduction is also recommended reading.