Marijn Faling1
If inclusive business is to realise wide and sustained development impacts, it is likely to depend on crowding in of other public and private actors. Assessing the contribution of inclusive business to crowding in is difficult because the phenomenon usually only manifests after project completion, and the complex operating environment complicates the process of evidencing a link between intervention and outcome. With donors placing increasing emphasis on demonstrating impact, innovative approaches to assess crowding in are needed. This article presents an adapted form of process tracing to assess the contribution of inclusive business to crowding in. It reports on the contribution of CREATE, an inclusive agribusiness project, to the crowding in of malting companies in Ethiopia’s barley value chain. Though predominantly focusing on demonstrating a programme’s contributions to crowding in, the article offers suggestions for how this process tracing-based exercise may support the fostering of inclusive agribusiness practices more broadly.
Crowding in, inclusive business, impact evaluation, process tracing, value chain intervention, Ethiopia.
Private sector engagement in development is gaining traction. This has encouraged the formation of inclusive agribusiness models – that is, ways of doing business that aim to improve the livelihood of smallholder farmers through integration in value chains in commercially viable ways (van Westen et al. 2019). Such approaches are often implemented in collaboration with other stakeholders in the value chain. The assumption is that through uniting the resources of public, private, and non-governmental stakeholders, development outcomes will exceed the outcomes that could be achieved by individual actors. Inclusive business models are assumed to enable wide-scale and sustained results (OECD n.d.; Hestad 2021), which not only benefit direct partners and target audiences but also bring changes in broader market systems (Schouten and Vellema 2019; Thorpe 2014).
Entities concerned with private sector development, such as the Donor Committee for Enterprise Development, Springfield Centre, and Building Effective and Accessible Markets (BEAM) Exchange, view crowding in as central to realising systemic change. Crowding in refers to the phenomenon whereby other public and private actors in the system adjust their practices in a manner that supports the intervention’s development objectives (Fowler and Dunn 2014; Nippard, Hitchins and Elliott 2014).
There are several features of crowding in that pose challenges for assessing it in evaluations. Usually when programmes close, systemic results like crowding in have only begun to materialise. This is because crowding in usually only manifests over longer time frames, beyond the temporal scope of an intervention (Kessler 2021). Validating the role of an intervention in stimulating crowding in requires the evaluator to be explicit about the link between intervention and outcome; this is in order to demonstrate that the effect is a consequence of the intervention and not something happening by chance or because of other developments (Mayne 2012). In the complex environment in which these programmes are implemented, convincingly demonstrating the contribution of a particular private sector development (PSD) programme to crowding in is difficult.
Meanwhile, donors and commissioners of impact evaluations are increasingly demanding an assessment of programmes’ contributions (Befani and Stedman-Bryce 2017). Besides serving accountability purposes, these evaluations may facilitate learning about effective processes of crowding in of inclusive agribusiness. Consequently, practitioners as well as researchers are piloting approaches to assess PSD contributions to processes of systemic change (Posthumus et al. 2020).
This article describes a collaborative exercise with the Community Revenue Enhancement Through Agricultural Technology Extension (CREATE) partnership, a collaborative private-sector engagement project in Ethiopia, during the period 2015–20. It focused on including smallholder farmers in the malt barley supply chain for beer production and the food market. Key partners included Heineken, the European Cooperative for Rural Development (EUCORD), the International Finance Corporation (IFC), and the Dutch Ministry of Foreign Affairs (MoFA). The project’s triple objectives were improving the wellbeing of 20,000 smallholder farmers, reducing reliance on imports, and contributing to food security. Its main interventions centred on local barley production and on connecting farmers to the value chain. CREATE claims to have contributed to the investments of two European malting companies that started operating malting plants in Ethiopia early in 2021. CREATE interpreted these as furthering its inclusive agribusiness objectives. Together with the CREATE partners, the collaborative exercise set out to find and assess evidence for this claim.
The approach was based on process tracing, adjusted in several ways to make it suitable for a relatively resource-constrained collaborative evaluation around future emergent outcomes. Adjustments included assessing the probative value of emergent future events instead of past events; and basing process tracing on existing data without additional data collection.
Section 2 of this article discusses the basics of process tracing, as well as the adjustments to tailor process tracing to assess contribution claims. Section 3 demonstrates how process tracing was applied to the case of CREATE. The exercise is discussed with conclusions in Section 4.
Process tracing is an approach of causal analysis used for in‑depth (multi-)case studies (Beach and Pedersen 2019). Although it has existed as a methodology in social sciences for some time, particularly history and political science, it is increasingly used in theory-based impact evaluation (Stern et al. 2012; Wauters and Beach 2018). Process tracing is used to explore and test causal inferences by critically analysing the sequence of events that have unfolded. It is based upon a mechanistic understanding of causality. It is a tool to unpack and critically assess a causal process consisting of interlinked mechanisms between an independent cause C (e.g. a PSD programme) and the dependent outcome O (e.g. the impact).
Mechanisms are often unobservable. Process tracing therefore distinguishes between hypotheses about causal mechanisms, and the observable and testable manifestations of the existence of those mechanisms in reality (Beach and Pedersen 2019). We cannot get full certainty about the existence of mechanisms, therefore process tracing helps to increase or decrease our confidence in the hypotheses about reality, in light of limited available information (Befani and Stedman-Bryce 2017; Fairfield and Charman 2017). The goal of process tracing is to approach the hypotheses like a detective and to look for the ‘evidence’ that convincingly demonstrates that a certain mechanism has taken place (Punton and Welle 2015a).
There are various forms of process tracing, depending on the nature and aim of the exercise (Beach and Pedersen 2019). Theory-testing process tracing assesses whether a hypothesised mechanism links intervention and outcome. Theory-building process tracing starts with empirics and is concerned with finding the mechanism that links intervention and outcome. Outcome-explaining process tracing involves collecting multiple causal mechanisms to explain a certain outcome of interest (Wauters and Beach 2018). Although all versions differ in their approach, they share some common characteristics in the way they look for and analyse pieces of evidence.
To determine whether the collected data would usefully serve as evidence, each piece of potential evidence is assessed according to the indicators of certainty and uniqueness. Certainty relates to whether we have to find the data for the hypothesis to be true, whereas uniqueness relates to whether there are alternative explanations for the presence of the piece of evidence (Beach 2017; Bennett 2015; Rohlfing 2012). The function of potential pieces of evidence for confirming or disconfirming hypotheses is determined by a combination of the certainty and uniqueness of evidence. The certainty of evidence is high when the evidence needs to be found to confirm our hypothesis. If certainty is low, evidence is not necessary to prove our hypothesis. The uniqueness of evidence is high when it is sufficient to confirm our hypothesis, whereas if the uniqueness is low, evidence leaves room for other explanations and does not prove that an intervention contributes to the impact (Beach and Pedersen 2019; Punton and Welle 2015b).
A single piece of evidence can underpin several hypotheses, while sometimes multiple data sources together form a piece of evidence. The evaluator should always question what the evidence found means, and whether it can be trusted. Imagine a farmer stating that their improved yields are the result of the support received from programme X. The reliability of this piece of evidence depends on the context and the motives of the farmer. If the farmer’s statement is the result of an interview by a practitioner from programme X, it is likely that the farmer does not want to disparage the programme. In that case, the evidence reveals little about the phenomenon of interest, and so additional evidence is needed to validate the hypothesis. Combined, the indicators of certainty and uniqueness and the assessment of reliability prompt the following questions for each piece of potential evidence (Beach and Pedersen 2019):
The exercise described in this article used process tracing in a customised manner. The article briefly discusses the steps involved and illustrates these in more depth in the subsequent section.2
Figure 1 sketches the steps of the exercise.
Figure 1 The steps of the process tracing exercise
Source: Author's own.
This section illustrates how the steps described above were used to demonstrate how the approach works for the evaluation of the contribution claim of CREATE about crowding in of the two malting companies to the benefit of smallholder inclusion.
CREATE aimed to commercialise farming based on contracts, supplying a package of high-yielding seed varieties and other agricultural inputs, alongside cultivation techniques such as row-planting and crop rotation. The objective of CREATE was to increase productivity and income – and thus wellbeing – of smallholder farmers; providing a secured market for their produce by connecting farmers to the malt barley value chain; contributing to food security; and reducing Heineken’s and the country’s reliance on imports (Heineken 2018). After the implementation period of CREATE, two malting companies decided to open malting factories in Ethiopia. CREATE claims that their activities have attracted these malting companies to invest. They consider this development to be a systemic effect of their project and supportive of realising the project’s inclusive agribusiness objectives.
The claim can be broken down into two separate overarching claims: (a) that the investments by Boortmalt and the Soufflet Group (Soufflet hereafter) can be causally linked to the CREATE project; and (b) that these investments support the original approach and objectives of CREATE towards inclusive agribusiness.
The following sub-sections follow the steps as described in Section 2 with discussion of each claim developed by the hypotheses. The article illustrates per hypothesis how we identified and assessed one of the pieces of evidence, and how the criteria of reliability were applied, what the evidence demonstrates, and what are the certainty and the uniqueness of the evidence. Table 1 illustrates the evaluation of all pieces of evidence.
The first part of the claim about CREATE’s role in attracting investments translates into the following hypothesis:
H1: CREATE has contributed to attracting investments of malting companies in Ethiopia’s malt barley value chain.
The hypothesis formed the basis for a discussion about how the partners perceived that CREATE had contributed to the crowding in of malting companies, and what data could be used as evidence to demonstrate CREATE’s contribution to this development. Three sets of potential evidence were identified in collaboration with the partners.
One of the pieces of data that partners identified is a public statement on video in which the Senior Operations Officer of the International Finance Corporation (IFC) explains that IFC has made an equity investment of US$20m in the malting company Soufflet Ethiopia, a subsidiary of the Soufflet Group. The reason for making the equity investment, as explained in the video, was partly because of the CREATE programme that through its positive results demonstrated the opportunities in terms of potential capacity of malt barley production in Ethiopia.
When assessing this piece of evidence, the first question concerns the reliability of the source. The video comes across as an authentic video in which we see the Senior Operations Officer of IFC explaining the reasons behind IFC’s equity investment. The video has been published by IFC, and therefore it is concluded that the data source itself can be trusted.
This leads to the second question of what the evidence tells us. The data are an indirect piece of evidence in the sense that it demonstrates that CREATE’s success in increasing the productivity of quality malting barley attracted investments that have financially supported the opening of Soufflet, one of the malting plants. Regarding certainty, we would not necessarily need to find this piece of evidence for the hypothesis to hold. IFC could have invested in Soufflet without publicly stating their rationale for doing so.
Further, technically Soufflet could have invested without an equity investment by IFC. Regarding the uniqueness of evidence, it needs to be certain that there are no plausible alternative explanations for finding this evidence that are unconnected to the contribution of CREATE. It could well be that IFC would praise CREATE, even without it being the real reason for making the equity investment. Because the evidence is neither certain nor unique, it is insufficient to confirm that CREATE motivated the malting companies to invest.
Therefore, in this exercise, the partners were brought together to discuss whether there would be additional evidence that could rule out any ‘bragging’ factor on the part of the IFC. The partners came up with an internal presentation by the Senior Operations Officer of IFC to the IFC board in which he presents the success of CREATE and raises the opportunity of investing in Soufflet following CREATE’s success. This piece of evidence is rather reliable, as an internal presentation would not be influenced by the potential need of keeping in mind a wider audience. It comes across as an authentic source as it contains the name of the official involved in CREATE and bears the IFC logo.
The presence of this piece of evidence makes it much less likely that the link between the equity investment and CREATE was just a promotional talk, and hence increases the uniqueness of the evidence. Consequently, combined, these pieces of evidence gave sufficient confidence that Soufflet has been attracted at least partly as a result of CREATE.
For the second part of the claim, it is necessary to assess whether the investments can be considered an indication of crowding in. Because this is an outcome that is only starting to emerge and has not come to fruition yet, it is not possible to know for sure whether crowding in will effectively occur. Instead, it is possible to test the probability that the investments can be considered as plausible indicators for the future crowding in. This also implies that these malting companies would need to support the inclusive agribusiness objective of improving the wellbeing of smallholder farmers:
H2: The investments by malting companies contribute to improving the wellbeing of smallholder farmers.
The partners identified several pieces of evidence that could potentially underpin hypothesis H2 (Table 1). The piece of evidence that could potentially strengthen this hypothesis entails data that indicate the existence and the nature of a follow-up barley value chain development project by Heineken, EUCORD, IFC, and one of the malting companies, Soufflet. The new programme, Barley Organization of Supply and Training in South East and Central Oromia (BOOST), will run from 2020 to 2023. The piece of evidence demonstrates how the project aims to enhance the productivity of farmers and the quality of their produce through access to improved seed varieties and other inputs, and by building capacity of barley producers. It aims to source 80,000 tonnes of barley annually from 55,000 mostly smallholder farmers (CREATE n.d.; Otuki 2021).
We first assessed the reliability of the source. It is an official project proposal, and there is public coverage of the project by several sources that are known to critically scrutinise the assumptions for investments in new development programmes. This means that the piece of evidence is considered as reliable.
What does the evidence reveal? Since it is a three-year project, the piece of evidence shows that in the coming years, Soufflet, together with other BOOST partners, will aim at improving the economic wellbeing of smallholder farmers, through increasing farmers’ productivity and product quality through access to improved seed varieties and other inputs, and technical and agronomic capacity-building measures. Their aim is to source 80 per cent from smallholder farmers. Although projects do not always manage to deliver the intended results, this is likely to do so, because of the involvement of CREATE partners and their experience and networks, which enhances the likelihood that BOOST will succeed. BOOST will not be a direct continuation of CREATE. Because the follow-up project is implemented in a different geographical location, it is unlikely that the follow-up project will claim outcomes that in fact have been produced by CREATE in the past, and not by BOOST’s support activities.3
Regarding certainty of the piece of evidence, given the widely held view that cross-sector partnerships are required to advance inclusive (agri)business approaches (Schouten and Vellema 2019), it is likely that Soufflet would engage in this collaborative initiative when it wanted to work on improving the wellbeing of smallholder farmers. There are no likely scenarios in which we would not find this evidence. Furthermore, since the malting company is partly reliant on the project for its malt barley supply, there are limited incentives for the maltster to leave the partnership. This means that the uniqueness of the evidence is high and therefore considered sufficient on its own to confirm the hypothesis.
It is useful to think about possible alternative explanations for the hypotheses, especially to put CREATE’s contribution in perspective. For instance, obviously CREATE has not been the only programme targeting Ethiopia’s barley value chain, and other value chain initiatives may have led to rising production and productivity as well. Furthermore, the government of Ethiopia has adopted a long-term strategy to promote the development of smallholder farmers and the agricultural sector, with the malt barley value chain as one of the target areas (Lavers 2011), and it may have had activities in the area that explain the outcome. Note that these alternative hypotheses are not necessarily rival hypotheses. Confirming either of the alternative hypotheses does not necessarily lead to disconfirming the main hypotheses about CREATE’s role in the process. A plausible alternative explanation about the crowding in of malting companies is therefore that other value chain initiatives have attracted Soufflet and Boortmalt to invest in Ethiopia’s malt barley value chain:
H3: Other initiatives have contributed to the crowding in of malting companies.
One of the pieces of evidence that could confirm this hypothesis is a 2019 Annual Report by the Agricultural Transformation Agency (ATA 2019), an initiative of the Ethiopian government to promote agricultural sector transformation. The report states how the efforts of the ATA have led to agreements with Soufflet and Boortmalt to establish malting plants. The website of the ATA mentions how the agreement signed with Boortmalt in 2017 grants the malting company land permits to build its factory (ATA 2017); and similar arrangements were made with Soufflet in 2018 (ATA 2018).
The reliability of this source of evidence is high, as the information is provided in official ATA communication channels published on their website. The evidence would demonstrate that the Ethiopian government has contributed to attracting investments by Boortmalt and Soufflet.
The certainty of the evidence is moderate. On the one hand, we would expect the government to report on its successes in terms of attracting foreign direct investments to the Ethiopian agricultural sector, especially given the government’s priority to boost agricultural modernisation. However, on the other hand, it would also be likely that the government would report on attracting investments in more general terms, meaning that we would not find explicit coverage of government spending at the level of detail of individual organisations. Therefore the certainty of this piece of evidence can be considered as moderate.
The uniqueness of this piece of evidence is rather high. It is very unlikely that an official government report would report on investments made by the government if there had been none. Furthermore, given the fact that the government administers all land in Ethiopia, it is very likely that the land permits would have been issued by the Ethiopian government. This means that this piece of evidence confirms the hypothesis that in addition to CREATE, other initiatives, more particularly initiatives from the government, have also contributed to the crowding in of the malting companies.
In addition, it is useful to verify alternative hypotheses to assess what the investments by the malting companies will lead to. Because the events that we are looking for have only begun to emerge, the alternative explanations are more likely, including the probability that the outcome develops in a different direction:
H4: The investments by malting companies are unlikely to realise any substantial improvements in the wellbeing of smallholder farmers.
For this hypothesis, it was not possible to rely on existing data from the partners. However, there is a variety of academic literature that discusses malt barley value chain projects in Ethiopia. One of these articles reports how the wellbeing improvements of malt barley value chain integration in Ethiopia seem to be dependent on the socioeconomic status of farmers; more specifically, that value chain integration initiatives tend to exclude older farmers and farmers who live in remote areas (Gebru et al. 2019).
The reliability of this piece of evidence is evaluated as high. The article is published in a well-known journal by a trustworthy publisher and every submission goes through a double-blind peer review process. The evidence would mean that the BOOST project might equally focus on farmers with certain economic and geographical characteristics, like large commercial farmers.
As it is not necessary to find this academic article for the hypothesis to be true, the certainty of the evidence that there will be no inclusive agricultural development is considered as low. This kind of finding about selective involvement of farmers in contract farming arrangements is usually highly dependent on the set-up of value chain interventions and the local context in which these interventions are implemented. We may have found this piece of evidence without the hypothesis needing to be true. We therefore assess the uniqueness of the evidence to be low as well. In conclusion, this piece of evidence is insufficient to confirm H4.
By systematically assessing the collected evidence for the different hypotheses, it can be concluded that we can partly confirm the claim that CREATE has contributed to the crowding in of malting companies (see Table 1). More precisely, we can claim with confidence that CREATE has contributed, alongside other factors, to the crowding in of Soufflet and that this likely contributes to part of CREATE’s objectives, including improving the economic wellbeing of smallholder farmers.
Table 1 Assessment of evidence
| # | Hypothesis | Evidence | Evidence of | Reliability | Certainty4 | Uniqueness5 | Evaluation6 |
|---|---|---|---|---|---|---|---|
| H1 | CREATE has contributed to attracting investments from malting companies Soufflet and Boortmalt in Ethiopia’s malt barley value chain | IFC statement regarding Soufflet | CREATE spurred investments that co‑facilitated the investments by Soufflet | High. Video comes across as authentic, publication by IFC emphasises reliability of the source. | Low | High | Partly confirms H1 |
| IFC internal presentation | |||||||
| Internal email conversation in which SECOBRA (barley breeding organisation of which Soufflet and Boortmalt are shareholders) requests to share right to the traveller barley variety introduced by CREATE with malting companies | Part of CREATE’s interventions (introduction of new seed varieties) are appreciated by malting companies | High. Data contains an original email conversation. No signs this conversation was manipulated in any way. | Low | Low | Does not confirm H1 | ||
| Quantitative data about CREATE’s successes in terms of rising production and productivity | Investment plans occurred sequentially after rising productivity of CREATE, as an indication that the willingness to invest occurred after the project has demonstrated positive results | Moderate. Success of CREATE is most strongly illustrated in project documentation, which may have used calculations that could exaggerate success of the project. However, additional sources confirm the rising production and productivity. | High | Low | Does not reject H1 | ||
| Signing MoU between Ethiopian government and malting companies | |||||||
| H2 | The investments by malting companies Soufflet and Boortmalt will likely improve the wellbeing of smallholder farmers | BOOST project coverage | Soufflet aims to improve smallholder integration into the value chain during 2020–23 | High. Other sources confirm existence and objectives of the partnership. | High | High | Partly confirms H2 |
| MSc thesis on supporting farmers in the malt barley value chain in Ethiopia | Boortmalt relies on a similar inclusive agribusiness model as Soufflet | Moderate. Authenticity of master’s theses is generally rather difficult to determine. | Low | Low | Does not confirm H2 | ||
| ATA 2019 Annual Report claims that malting companies’ investments will improve the livelihoods of 10,000 farmers | Government is optimistic about the impact of malting companies on farmers’ wellbeing | High. Official report published on ATA’s website. | Low | Low | Does not confirm H2 | ||
| H3 | Other value chain initiatives have contributed to the crowding in of malting companies | ATA report covering government’s efforts around attracting malting companies | Ethiopian government has contributed to attracting investments Boortmalt and Soufflet | High. Official government documentation published on ATA website. | Moderate | High | Confirms H3 |
| Project page Sourcing for Growth (S4G) partnership | Other initiatives have contributed to improving productivity and quality of malt barley | High. Other sources confirm existence and objectives of the partnership. | Low | High | Confirms H3 | ||
| H4 | The investments by malting companies Soufflet and Boortmalt are unlikely to realise any substantial improvements in the wellbeing of smallholder farmers | Academic article discussing how wellbeing improvements through barley value chain integration in Ethiopia depend on economic status farmer | BOOST project likely to focus on farmers with certain economic and geographical characteristics, excluding older and more remote farmers | High. Academic articles that go through peer-review process may be expected to contain reliable information. | Low | Low | Does not confirm H4 |
Source: Author's own.
This article reports on an exercise to assess the reliability of the pieces of evidence to support a claim that a programme contributed to systemic changes beyond the temporal and spatial boundaries of the programme. Based on a collaborative approach to identify and critically assess the evidence, it was possible to confirm the hypotheses about the crowding in of at least one of the malting companies. This means that following this process tracing approach, CREATE can claim with confidence that the project has contributed to the crowding in of other actors, more specifically the malting company Soufflet, into CREATE’s inclusive agribusiness approach. The establishment of the subsidiary Soufflet Ethiopia is likely to contribute to improving the wellbeing of smallholder farmers. The exercise has helped to advance insight about the likelihood that the process of development as pursued by CREATE will continue beyond project termination.
Though far from a done deal, this scrutiny of the pieces of evidence helped us to become more precise and certain about the contribution of an inclusive agribusiness. The guidance of process tracing helps to approach the formulation of contribution claims and the selection and assessment of evidence in a structured way, by making use of the criteria of necessity that the piece of evidence would be present and sufficiency of the evidence for the claim to be true. As such, the application of this approach enables the evaluator to increase the robustness and conceptual precision of contribution claims (Befani and Stedman-Bryce 2017).
The exercise demonstrates that process tracing, although frequently presented as time- and resource-intensive (Hay 2016), can also be undertaken in a simpler way. By making use of available evidence and expert judgements of practitioners to identify this evidence, the approach of process tracing becomes achievable, even with limited resources. This approach also opens the door to more robust collaborative evaluation approaches. An often-cited risk with collaborative evaluation approaches is that the evaluator becomes too engaged, leading to bias in the findings (Mapitsa and Chirau 2019; Braskamp, Brandenburg and Ory 1987). The explicit guidance offered by process tracing approaches helps to collect and assess data offered through a process tracing approach, and functions to improve independence and critical scrutiny when assessing contributions collaboratively.
Although an exercise such as this one seems to be capable of enhancing our confidence in a programme’s contribution to crowding in, this is just one experience of how to use a process tracing approach in a collaborative context. Much work remains to be done. While the approach has demonstrated (a) that CREATE has contributed to the crowding in of malting companies, and (b) that one of the malting companies is likely to contribute to continuing and widening the benefits for smallholder farmers, this exercise tells us little about the precise pathways and activities through which CREATE has fostered these investments. This is an important void that needs to be addressed.
The central objective of process tracing is to unpack the causal process that links cause and outcome, by looking for evidence along the causal chain. The unpacking of the causal process between intervention and outcome can be done more granularly than has been possible in this article. Tracing the causal process is particularly important for monitoring and learning processes, as these require timely feedback on progress and direction of programmes, and an understanding of the processes through which (combinations of) strategies and processes contributed to the results (Rogers and Macfarlan 2020). More granular insights would support practitioners in developing a sensitivity to recognise crowding in, to help them strategise to reach outcomes that are beyond their direct sphere of influence.
The exercise presented in this article could serve as a first iteration and starting point to further understand the pathways through which CREATE has triggered crowding in (Taylor, Torugsa and Arundel 2018). As concrete follow-up to this research, existing theoretical knowledge about pathways towards crowding in could be used to propose new hypotheses that are empirically testable in a subsequent round of process tracing, with the objective of better understanding the particular pathways through which crowding in can be fostered and nurtured.
Furthermore, incorporating exercises like these in follow-up programmes will help to broaden the regular focus on outputs of activities to include the more systemic outcomes of programmes in monitoring and evaluation efforts. Becoming aware of the signs of systemic changes will help practitioners to track processes outside regular result frameworks and log frames. Articulating and critically evidencing a programme’s contribution claim enables practitioners and evaluators to set boundaries of what needs to be focused upon, both programmatically and in terms of monitoring. Systemic changes can as such be incorporated into programme management so that it helps practitioners to continue fostering this process, in order to nourish the continuation and widening of inclusive agribusiness practices.
* The author thanks the Community Revenue Enhancement Through Agricultural Technology Extension (CREATE) partners for their invaluable contribution to the research. The research was made possible through the grant of the Dutch Ministry of Foreign Affairs (MoFA) to the Partnerships Resource Centre. The author furthermore thanks the editors Sietze Vellema and Giel Ton for their feedback, which helped improve the focus and approach of the manuscript.
1 Marijn Faling, Partnerships Resource Centre (PrC), Erasmus University Rotterdam, Netherlands.
2 The exercise was funded by MoFA to explore innovative approaches of enhancing public accountability and facilitate learning processes regarding the ways in which private sector engagement modalities contribute to systemic change. The partnership was selected as a typical case; selection was based on the alleged contribution of CREATE to the crowding in of malting companies. The exercise was undertaken in 2018–19 in collaboration with representatives from the main partners engaged in CREATE (MoFA, EUCORD, Heineken). It consisted of two rounds of interactive workshops, and various bilateral conversations with the individual partners.
3 CREATE was implemented in Arsi, West-Arsi, and Bale zones; the BOOST project will be implemented in Assela zone.
4 The criterion of certainty relates to whether we have to find the evidence for the hypothesis (H) to be true.
5 This criterion relates to whether there are alternative explanations for the presence of the evidence.
6 (Dis)confirmation of the H is based on the certainty and uniqueness of the evidence.
ATA (2019) Annual Report on Transforming Ethiopian Agriculture, Addis Ababa: Agricultural Transformation Agency
ATA (2018) ‘Malteries Soufflet Signs a Land Lease Agreement with Industrial Parks Development Corporation’, Ethiopian Agricultural Transformation Agency, 13 June (accessed 10 July 2021)
ATA (2017) ‘Landmark Lease Agreement Signed for International Investment in Malting Factory’, Ethiopian Agricultural Transformation Agency, 24 November (accessed 10 July 2021)
Beach, D. (2017) ‘Process-Tracing Methods in Social Science’, Oxford Research Encyclopedias of Politics, 25 January (accessed 14 October 2021)
Beach, D. and Pedersen, R.B. (2019) Process-Tracing Methods: Foundations and Guidelines, 2nd ed., Michigan MI: University of Michigan Press
Befani, B. and Stedman-Bryce, G. (2017) ‘Process Tracing and Bayesian Updating for Impact Evaluation’, Evaluation 23.1: 42–60
Bennett, A. (2015) ‘Appendix: Disciplining our Conjectures: Systematizing Process Tracing with Bayesian Analysis’, in A. Bennett and J.T. Checkel (eds), Process Tracing: From Metaphor to Analytic Tool, Cambridge: Cambridge University Press
Braskamp, L.A.; Brandenburg, D.C. and Ory, J.C. (1987) ‘Lessons About Clients’ Expectations’, in J. Nowakowski (ed.), The Client Perspective on Evaluation: New Directions for Program Evaluation, No. 36, San Francisco CA: Jossey-Bass
EUCORD (n.d.) BOOST Ethiopia, CREATE (accessed 24 September 2021)
Fairfield, T. and Charman, A. (2017) ‘Explicit Bayesian Analysis for Process Tracing: Guidelines, Opportunities, and Caveats’, Political Analysis 25.3: 363–80, DOI: doi:10.1017/pan.2017.14 (accessed 31 August 2021)
Fowler, B. and Dunn, E. (2014) Evaluating Systems and Systemic Change for Inclusive Market Development: Literature Review and Synthesis, Leveraging Economic Opportunities Report 3, Washington DC: United States Agency for International Development
Gebru, K.M.; Rammelt, C.; Leung, M.; Zoomers, A. and van Westen, G. (2019) ‘Inclusive Malt Barley Business and Household Food Security in Lay Gayint District of Northern Ethiopia’, Food Security 11.4: 953–66
Hay, C. (2016) ‘Process Tracing: A Laudable Aim or a High-Tariff Methodology?’, New Political Economy 21.5: 500–4
Heineken (2018) Growing with Ethiopia, CREATE (accessed 24 September 2021)
Hestad, D. (2021) ‘The Evolution of Private Sector Action in Sustainable Development’, IISD Earth Negotiations Bulletin: Brief 12, Winnipeg: International Institute for Sustainable Development (accessed 31 August 2021)
Kessler, A. (2021) Assessing Systemic Change: Implementation Guidelines for the DCED Standard, Donor Committee for Enterprise Development
Lavers, T. (2011) The Role of Foreign Investment in Ethiopia’s Smallholder-Focused Agricultural Development Strategy, LDPI Working Paper 2, The Hague: Land Deal Politics Initiative (accessed 31 August 2021)
Mapitsa, C.B. and Chirau, T.J. (2019) ‘Institutionalising the Evaluation Function: A South African Study of Impartiality, Use and Cost’, Evaluation and Program Planning 75: 38–42
Mayne, J. (2012) ‘Contribution Analysis: Coming of Age?’, Evaluation 18.3: 270–80, DOI: 10.1177/1356389012451663 (accessed 8 October 2021)
Nippard, D.; Hitchins, R. and Elliott, D. (2014) Adopt-Adapt-Expand-Respond: A Framework for Managing and Measuring Systemic Change Processes, Springfield Centre Briefing Paper, Durham: Springfield Centre (accessed 9 September 2021)
OECD (n.d.) ‘The Holistic Toolbox for Private Sector Engagement in Development Co-operation’, Private Sector Peer Learning Policy Brief 1, Paris: Organisation for Economic Co-operation and Development (accessed 1 September 2021)
Otuki, N. (2021) In Ethiopia, Linking Farmers to Prosperity, International Finance Corporation (accessed 24 September 2021)
Posthumus, H.; Shah, R.; Miehlbrandt, A. and Kessler, A. (2020) A Pragmatic Approach to Assessing System Change, Durham: Springfield Centre (accessed 1 September 2021)
Punton, M. and Welle, K. (2015a) Straws-in-the-Wind, Hoops and Smoking Guns: What Can Process Tracing Offer to Impact Evaluation?, CDI Practice Paper 10, Brighton: Institute of Development Studies (accessed 8 October 2021)
Punton, M. and Welle, K. (2015b) Applying Process Tracing in Five Steps, Annexe to CDI Practice Paper 10, Brighton: Institute of Development Studies (accessed 1 September 2021)
Rogers, P. and Macfarlan, A. (2020) An Overview of Monitoring and Evaluation for Adaptive Management, Monitoring and Evaluation for Adaptive Management Working Paper 1, BetterEvaluation (accessed 1 September 2021)
Rohlfing, I. (2012) Case Studies and Causal Inference: An Integrative Framework, Basingstoke: Palgrave Macmillan
Schouten, G. and Vellema, S. (2019) ‘Partnering for Inclusive Business in Food Provisioning’, Current Opinion in Environmental Sustainability 41: 38–42, DOI: 10.1016/j.cosust.2019.10.004 (accessed 8 October 2021)
Stern, E. et al. (2012) Broadening the Range of Designs and Methods for Impact Evaluation, London: Department for International Development
Taylor, R.; Torugsa, N.A. and Arundel, A. (2018) ‘Leaping Into Real-World Relevance: An “Abduction” Process for Nonprofit Research’, Nonprofit and Voluntary Sector Quarterly 47.1: 206–27
Thorpe, J. (2014) Business and International Development: Is Systemic Change Part of the Business Approach?, IDS Evidence Report 92, Brighton: Institute of Development Studies (accessed 8 October 2021)
van Westen, G.; Mangnus, E.; Wangu, J. and Worku, S.G. (2019) ‘Inclusive Agribusiness Models in the Global South: The Impact on Local Food Security’, Current Opinion in Environmental Sustainability 41: 64–8, DOI: 10.1016/j.cosust.2019.11.003 (accessed 8 October 2021)
Wauters, B. and Beach, D. (2018) ‘Process Tracing and Congruence Analysis to Support Theory-Based Evaluation’, Evaluation 24.3: 284–305, DOI: 10.1177/1356389018786081 (accessed 8 October 2021)
© 2022 The Author. IDS Bulletin © Institute of Development Studies | DOI: 10.19088/1968-2022.108
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC BY-NC), which permits use, distribution and reproduction in any medium, provided the original authors and source are credited, any modifications or adaptations are indicated, and the work is not used for commercial purposes.
The IDS Bulletin is published by Institute of Development Studies, Library Road, Brighton BN1 9RE, UK. This article is part of IDS Bulletin Vol. 53 No. 1 February 2022 ‘Theory-Based Evaluation of Inclusive Business Programmes’; the Introduction is also recommended reading.