Dee Jupp,1 David Korboe2 and Tony Dogbe3
Abstract
In the external evaluation of development interventions, beneficiaries are often involved to define priorities, and provide feedback and evaluation. This generally uses ‘invited spaces’ such as facilitated community meetings, focus groups, mobile phone-enabled feedback, and social audits. Where external evaluation must be both independent and separate from the project’s own learning and adaptation processes, this can pose challenges. This article asks whether informal immersion in beneficiaries’ ‘own space’ can provide insights beyond ‘invited spaces’ to enhance our understanding of how people experience development interventions, particularly where these interventions are integrated and complex. The article describes the inclusion of one type of immersion research, the Reality Check Approach (RCA) within the suite of qualitative and quantitative methods used in the longitudinal (external) impact evaluation of the Ghana site under the Millennium Villages Project (MVP). The RCA in this evaluation provided a means to spend concentrated time in beneficiaries’ ‘own space’ without a project (theory-based) evaluation lens.
Keywords: immersion research; Reality Check Approach; beneficiary ‘own space’.
1 Introduction
Cooke and Kothari (2001) noted three types of participation ‘tyranny’: the imposition of external systems of participation which override existing ones; the tyranny of group dynamics which favour consensus and are subject to co-option; and the tyranny of the approach which has squeezed out other ways of involving people and understanding their situation. These criticisms were subsequently strongly refuted in the context of participation as a transformative and empowering process, citing a growing body of evidence of the use of a range of innovative participatory approaches (Hickey and Mohan 2004). Unless specifically designed to provide a third-party learning function, external evaluation4 is often by its nature required to remain outside
of the project, in order to provide an independent assessment of both the impact and processes described by the project design and business case. Such separation constrains the use of genuinely empowering participatory processes in evaluation, as these might be expected to influence the process and outcomes of the project.
The conventional means to involve beneficiaries in such external evaluations are typically through ‘invited spaces’ (Gaventa 2006) such as focus groups, interviews, specially convened participatory fora (e.g. as used in a range of participatory approaches such as participatory rural appraisal (PRA), social audits or Most Significant Change (MSC) workshops) – which mostly use project evaluation frameworks as a basis to solicit assessment. Since external evaluation is constrained by the requirement of non-interference, there is then little but ‘invited space’ to employ to involve beneficiaries. Rarely do external evaluations draw on ‘popular space’ (defined as arenas in which people come together at their own instigation – for example, to protest or for self-help and solidarity) (Cornwall 2004). This is often because beneficiaries continue to lack the agency and voice required to occupy ‘popular space’ (Mahmud 2007).
Given these limitations, there is more risk that the criticisms raised by Cooke and Kothari (2001) may be valid in external evaluation situations than, say, in adaptive project design and implementation. In such circumstances, immersion research may provide a middle ground between people’s participation in ‘invited spaces’ and ‘popular space’, by enabling insights to be gathered into how people experience change. Does this have the potential to provide a sort of ‘passive participation space’ where the costs of participation for people are minimised and where the informality of the arrangement enables engagement with those who often eschew (or are excluded from) more public types of engagement?
The Reality Check Approach (RCA)5 is one form of immersive research which has many applications, but in this case was integrated within the mixed methods longitudinal and external impact evaluation of the MVP in northern Ghana. The RCA involves researchers living with people in their own homes for a few days and nights in order to ‘hang out’ (Geertz 1998) with them, their neighbours, and others they meet during the course of ordinary days and nights. This informal interaction in people’s ‘own space’ is intended to reduce power inequalities between people and researchers (with researchers ‘experiencing’ day-to-day life) and provides opportunities to combine listening with observation and direct experience.
The RCA was included in the suite of mixed methods for the evaluation of the MVP as a means to get closer to understanding people’s perspectives and experience of change throughout the five years of the project. In this sense it was exploratory, but it was anticipated in the design and sequencing of the immersions with other evaluation approaches that it would also be able to provide alternative and emic narratives around change and the drivers of change. It was considered that this approach might complement the use of PRA with convened groups.
Whilst the PRA sessions were well facilitated and participatory, they were nevertheless ‘invited spaces’ (albeit at village level), to which the same cohort of participants were invited at each phase of the evaluation, and participatory analysis was largely focused on project processes and interpreting emerging findings on effects. By contrast, the RCA was not framed within an evaluation or project context but rather one where researchers wanted to immerse in day-to-day life and gather insights from this immersion. Further, in this evaluation, all researchers but three team leaders did not know they were in any way associated with the project6 and were purposely primed to immerse in communities with open minds, and to focus only on how people experienced change and how they attributed change. This provided something similar to a double-blind approach, in that not only were the field researchers independent of the project, but the people they lived with, and interacted with, also did not filter their views through a project evaluation lens.
Immersion can add value compared with many other qualitative approaches, as it provides opportunities for immediate triangulation – comparing what people say has changed with direct observation and the researchers’ own experience, for example. Those who habitually undertake immersion studies also note during critical reflection post-immersion that engaging with people through shared experience and using the context to stimulate conversation leads to a perception of greater authenticity in the findings (and by inference, a reduction particularly in social desirability and habituation bias). Conversations are natural, two-way, and can be returned to as new insights emerge over the course of the immersion. Immersion offers opportunities to interact with those who may eschew ‘invited spaces’ or ‘popular spaces’, or who may not be able to participate in such spaces, thereby enabling a wider net of perspectives for triangulation. Living with people in their own space leads to opportunities to build trust which cannot be so easily built in ‘invited spaces’ with fixed meeting places, time constraints, and external agendas. It provides opportunities to explore unexpected insights and to iterate findings through being ‘on the spot’ over time,7 including day and night, and on different days of the week. It enables people to participate without disruption to their daily activities and income-earning as researchers accompany and participate in chores too.
Immersion research nevertheless has limitations, including scale, selection bias, and researcher bias. In the case of the MVP Impact Evaluation, the RCA researchers lived with only 20 households, about three households in each of six villages (four ‘treatment’ Millennium Villages (MVs) and two comparison villages). However, households in northern Ghana are often quite large, usually multi-generational, and on each of the three occasions, the researchers lived with the households and were able to ‘hang out’ and have insightful interactions with an estimated 1,500 people comprising a wider demography including neighbours, local tradespeople, local leaders, and service providers such as school teachers and health staff. Given the size of the MVP, which covered 34 villages and 30,000 people, the RCA only comprised a small number of households in context-specific locations.
The immersion was purposely with poorer households, although, as noted, there were important interactions with other socioeconomic groups and service providers. To mitigate some aspects of selection bias, researchers navigated the discussions with villagers and households on the proposed living arrangements themselves and avoided gatekeepers or intermediaries. Families were purposely located both in the centre and periphery of villages. Each immersion was also quite short (four nights and five days so over the course of the five years, 12 nights with each family).
Other limitations inherent in immersion research such as concerns about the positionality of researchers and differential power relationships between researcher and families, and within the family and community were discussed extensively during the training of researchers where means to mitigate such limitations were practised through simulations. Plus, like all ethnographic-based work, efforts were continuously made to critically review practice and exercise reflexivity. The RCA is always carried out in teams rather than as individuals (although immersion is individual) in order to reduce researcher bias, to widen the net of perspectives, and to ensure critical reflexive practice.
2 Insights gained from immersion in beneficiaries’ ‘own space’
Both the MVP’s own monitoring system and that adopted by this third-party, external evaluation sought to measure achievement against pre-determined indicators, primarily the Millennium Development Goals (MDGs), but also those presumed to represent the results of the package of interventions. This meant that the emphasis was on measuring what was intended and was inherently a normative and, in this case, a purely externally constructed project view.
As mentioned above, the RCA interactions eschewed a project lens and looked at change more broadly. In the third and final immersion in 2017 (six months after the official closure of the MVP), the RCA researchers used the ‘hanging out’ methodology to purposely reflect with the families they had lived with over the previous five years as well as with the range of other study participants as described above, on change (good change and bad change) and the perceived drivers of those changes. As this was the third immersion, researchers were able to share the experience of many of these changes through using local transport and roads, accompanying children to school, accompanying patients to health facilities, eating with families, taking produce to market, experiencing change in agricultural practices, sleeping in their homes, using water and sanitation facilities, and so on.
The insights generated through informal conversations, shared experience, and observations were brought together immediately following the immersions during intensive sessions with each village research team in turn, where details of the changes and their significance to people were compiled without filter and critical reflexivity was practised. These ‘downloading sessions’ were followed by all village research teams coming together for a sense-making workshop where findings were compared across locations and study participants. Immersion in the data set comprising records of conversations, observations, personal experience, photos, and artefacts led to a search for emergent themes which were coded and charted using conventional grounded theory approaches, with special attention paid to disaggregating themes by study participant categories (primarily gender, age, and socioeconomic categories8).
Interestingly, the three most significant changes identified across categories of adults were the same, although the reasons for the significance were often nuanced. Furthermore, people rarely noted any other change beyond these three. Based on multiple conversations and frequency of mention, the three key changes for adults of all ages and across socioeconomic categories were: (1) electricity connection, (2) cowpea cultivation, and (3) increased mobility and connectivity.
The MVP had prioritised specific ‘proven’ interventions, including in health (new and rehabilitated health centres and a range of preventative health-care interventions), schooling (new and rehabilitated schools, teachers, and resources), road improvements, and agriculture (introduction of maize, provision of inputs such as seeds and artificial fertiliser, and tractor hire) as an engine for economic growth. Significantly, people’s views of what was significant change embraced social benefits as much as the economic benefits emphasised by the project. So, for example, the benefits of electricity connection were largely seen in terms of social benefits – household connections for home lighting, TV, and easing the burden of milling for home consumption, as well as ensuring that social services were resourced because teachers and nurses would not stay in villages without electricity. Cowpea cultivation was significant, not only because it provided additional cash (in contexts which increasingly require cash) in an otherwise unproductive season, but as a means to halt the seasonal migration for work which split families for many months every year. In terms of mobility and connectivity, it was the increase in what are locally referred to as motorkings (small three-wheeled motorised trucks used to transport goods and people) and private motorbikes that was noted and primarily viewed in terms of time-saving and convenience for recreational as much as economic purposes. The significance of the increased use of mobile phones was also couched mostly in terms of social connectivity, and only secondarily as an opportunity to access mobile money, and not in terms of accessing work or trade opportunities.
The following examines each of these three key changes in more detail, based on insights from the immersion in beneficiaries’ ‘own space’, how these added to the external evaluation, and how they could have been useful for design of the intended further roll-out of the MVP, had this actually happened.
3 Differing priorities and views of integrated development
Under the MVP, electrification was framed in terms of an engine for economic development and more efficient service provision, and the project lobbied the Electricity Company of Ghana Ltd to prioritise the MVs in their National Electrification Scheme (NES) and in their rural electrification plan. Through conversations during the immersion, it was clear that connection to electricity was perceived as a major sign of development by adults of all ages, as well as children, with comments such as ‘We are like towns now’. Families enjoyed light at night (mostly keeping the lights on overnight, saying that they liked to feel safe, to attend to children in the night, and that keeping lights on at night had become the village ‘norm’), better milling facilities for personal use (not commercial use), and buying cold drinks from local shops.
But, without doubt, the most significant change referred to widely and observed in the final immersion was access to TV. The experience of walking round the villages at night in 2017 was very different from previous immersions in 2013 and 2015, when women and children went to bed soon after dusk, and men tended to gather in bars or small shelters to chat, but also retired to bed early. Many houses now had their own TV, usually bought second-hand with cash (around GHS150 (£24), satellite dish around GHS250 (£40)). People shared that it gave them something to enjoy, especially in the evenings, as illustrated by comments made by the elderly aunties in one home: ‘We now play in the evenings and go to bed later’. Others noted that it made them feel ‘connected’ with the world outside their village.
The change also brought concerns. Many parents and school teachers worried about children watching TV late and oversleeping or dozing in class. Whilst some families liked neighbours coming around to watch programmes, others resented their loss of privacy. Bill-payers (both men and women) were increasingly concerned about the costs of their electricity, and many shared that they were fearing disconnection, something they thought they would feel acutely, having become accustomed to lights and TV. Whether they were conservative in their use of electricity or not, the costs of connection (GHS40–120), electricity consumption (minimum GHS2–15/month), and the need for bulbs (GHS2.5 each, which last about a month) were considered major additional cash expenses.
As well as the largely social benefits of electricity at household level, the teachers and nurses interacted with during the immersion explained that they would not have relocated to the villages if there had been no electricity, and where quarters had been provided without reliable electricity connections, these were not occupied. In their words, it meant that they could ‘have fridges, satellite TV, and computers’,9 with some saying that this was essential as ‘social life in the village is difficult’.10 The immersion confirmed that they had very little interaction with villagers outside of school or work.
A number of new mills have been established or old diesel generator-powered mills have been converted, mostly through non-metered (illegal) connections. A project lens might conclude that this was demonstration of enhanced economic activity, but spending time at these mills ‘hanging out’ revealed that only small quantities are brought for milling for personal use just as before, and not for processing for on-selling by the customer. Women shared that ‘electric mills grind flour softer and nicer’11 and there is ‘no smell of diesel’12 compared to the past. Some shops had invested in fridges, but observation indicated that these are always filled exclusively with cold drinks.
In terms of raising agricultural productivity for economic gain, the MVP had decided a priori based on expert advice to concentrate on four promising crops; maize, rice, acacia, and mango. Extension programmes were designed to promote the cultivation of these crops including demonstration plots, subsidised seed and saplings, and new fixed-price tractor services. However, it was cowpea cultivation that was cited by men and women as the second most important change and the most important economically they had experienced, as illustrated by the quote from a young farmer in 2017: ‘If you are not part of the cowpea business you are dead already’.13 It was very evident from the 2017 immersion that the practice had grown phenomenally since 2013, with observation of stocks of cowpeas in people’s homes and the addition of cowpeas to TZ14 to at least one of the meals taken each day, when previously researchers and families had eaten TZ alone.
In conversations, men and women were clear that this has not been driven by external development programmes, but through farmer-to-farmer diffusion based on a single role model (whom different people in villages far from one another cited) who, in 2013 demonstrated hugely increased yields using insecticides and weed killer. An external view suggests the MVP’s promotion of maize may have facilitated this change because maize has a shorter crop period compared with millet, the traditional crop commonly observed in immersions in 2013, allowing for a second crop (cowpeas) to be grown. However, people themselves felt that increased market availability of agricultural inputs were what enabled the change. Immersion in the villages indeed demonstrated the rapid rise in retail of agro-chemicals with new outlets along roadsides, market stalls in weekly markets which had not been there in 2013 and 2015, and a proliferation of discarded agro-chemical containers littering paths to fields surrounding the villages. The substitution of traditional millet growing with maize was also explained by people in terms of increasingly reduced millet yields, and families shared that it was also welcomed because millet was considered a labour-intensive crop, especially at maturation, when flocks of birds have to be kept away. The ease of access to agro-chemicals in local markets also meant that farmers could grow maize and cowpeas on their own without needing others to help them with clearing the soil and weeding, as they had done with other crops (requiring time-consuming, traditional reciprocal arrangements), or having to pay for labour. Those in both project and comparison villages, who indicated that they felt better off than when researchers met them before in 2013 and 2015, attributed this almost entirely to cowpea cultivation, and many shared that they had been able to purchase assets such as tin roofing, extended their houses or, of course, bought TVs and motorbikes with the profits.
Economic gains were only part of the reason that people felt cowpeas were so significant. In some villages, cowpea cultivation was said to have helped reverse the traditional seasonal migration of both men and women that used to take place during the dry season. People shared, ‘There is no need to go South now as the income from cowpeas is good and we have electricity’.15 This was valued as a means to keep the family together and to avoid the stress created by concern for the safety of those migrating. Women shared that because they were key in harvesting and processing cowpeas, they were able to use small stocks as their own personal savings, and to sell small quantities for cash for items such as personal toiletries, which had not been possible before.
In addition to electricity and the cowpea cultivation discussed above, people also cited mobility and connectivity as a key change in the area. The MVP framed improvements in road infrastructure as part of its integrated approach, in terms of providing all-weather connectivity to make roads useable by haulage trucks to convey agricultural products out of the MV. Much like electricity provision, the MVP did not directly invest in this but lobbied the national and local government to prioritise the construction and maintenance of a number of key roads servicing MV areas.
Immersion in villages revealed a conundrum, however. Men and women alike highlighted increased mobility as a key change – like other changes viewed through a project lens, this could be attributed to the improved roads. But conversations based on shared experience between families and researchers often turned to the poor state of roads which had been rehabilitated with many collapsed culverts and dangerously deep potholes. One particular road was described as ‘much worse than before’. Insights were gained by observation of the massive increase in motorkings and private motorbikes. The increase was observable in both project and comparison villages. The source of these bikes was a British–Ghana venture established in Tamale in 2007, with Chinese expertise to build and sell motorkings initially, and later motorbikes.
People explained, and researchers observed and experienced, that both types of vehicle can deftly navigate broken culverts and deep potholes and can access quite rudimentary paths. People also explained that the proliferation was enabled by the substantial increase in disposable cash (such as from profits from cowpeas) and easy credit terms provided by the local company. Men and women indicated that these vehicles were not expensive to run and provided a much more flexible and reliable option than the market trucks or pushbikes of the past, being both convenient and time-saving.
Living in the homes of families who had purchased motorbikes in the year leading up to the third and final immersion enabled observation and discussion around the use of motorbikes, augmented by observation and chats with others. Most journeys were for social reasons – to meet up with friends and relatives, recreational, and accessing fields – and less used for accessing markets for buying or selling.
The anticipated increase in haulage trucks envisaged by the MVP was not apparent, except on a few short stretches of the roads which were passable. On one major artery which had been rehabilitated through MVP efforts (as confirmed from signage seen by researchers), subsequent deterioration was so bad that market trucks could no longer pass, and the weekly market had halved in size as people preferred to access a different market using a different route. Women market-sellers from outside and families we lived with predicted the market would soon cease altogether.
In addition to physical mobility, people shared that they felt more connected due to mobile phones and this was almost entirely viewed in terms of social relationships. Tigo, the preferred service provider, was only in two MVP locations where RCA immersion was undertaken, and which initially had a partnership with MVP and installed masts in these two villages. In other villages, better signals were obtained from Vodafone and MTN. Economic benefits were only cited in areas with provision of good MTN signals which enabled access to mobile money.
4 Discussion
The MVP describes itself as bottom up, participatory, and community-led: ‘Participatory, community-led decision-making is central to the way Millennium Villages work and is also fundamental to sustainability’.16 As noted by Carr, however:
Descriptions of the MVP as a ‘bottom-up’ approach are questionable, given the project’s reliance on pre-conceived definitions of problems and pre-packaged solutions to address poverty at the village level. These pre-conceptions present serious challenges to understanding the actual problems faced by the people living in these villages (2008: 334).
The MVP did indeed make use of some ‘invited spaces’ to refine interventions. During the project, there was one example of ‘popular space’ use comprising a protest made by farmers over a failed scheme to provide fertilisers for maize production. These were provided too late, yields suffered, repayments could not be made, and farmers were harassed. Many farmers did not want to join the scheme the following year and it was dropped. Although the project convened community and group meetings, these were largely to provide information and organise interventions, and there was no apparent mechanism within the project to gather ongoing feedback or beneficiary assessment – nor obvious attempts to empower beneficiaries to actively participate and influence the implementation of the project. Had the MVP adopted a more effective participatory approach to project management and adaptation, there may have been opportunities for the external evaluation team to have interacted with community groups and to learn about the progress and impact of the programmes through their lenses.
As the external evaluation used theory-based impact evaluation (White 2009) principles, this meant achievements were assessed primarily in terms of project objectives and its underlying Theory of Change. The evaluation methods used mostly involved ‘invited spaces’, albeit at village locations (such as household surveys, and facilitated group meetings). The inclusion of immersion research complemented this approach by not being constrained by the evaluation framework. Instead, it used a simple ‘hanging out’ methodology in beneficiaries’ ‘own space’, together with grounded theory. In doing so, it could provide insights into what changes were important to people, such as highlighting how social meaning was mostly overlooked in project theory and design. It also prompted areas of enquiry and analysis which might have been neglected without the combination of conversations, observation, and experience inherent in and valued by ‘hanging out’.
One important insight is that the three key changes highlighted by people above (electricity, cowpea cultivation, and mobility) were not directly funded by the MVP. From the outset, the MVP had intended to leverage electricity connections and improved roads as important enabling elements of the integrated approach. The MVP did not promote cowpea cultivation and did not noticeably react to the (concerning) rise in agro-chemical use. In terms of wellbeing and what development means, people also shared very different views than those perpetuated by the MVP; having a light on all night, owning a TV, a motorbike, and phones, and being able to consume cold drinks made important differences to the quality of people’s lives. These also provided opportunities for social interaction – which all constitute significant contributions to a sense of wellbeing.
The experience of this immersion research suggests the desirability of including processes to connect with people’s experience of the interventions made in their name on a regular basis. Had the immersion research been undertaken as part of project implementation, and not only as an explicitly independent external evaluation, the project could have benefited from being continuously fed these sorts of insights to improve the adaptation and sequencing of interventions within the integrated programme. Given the original intention that the external evaluation would determine whether further investment and up-scaling in the MVP should be considered, the immersion research could have also provided important insights into future programming by unpacking the normative input-driven model, and offering people’s perspective unshackled from sponsor bias.
Whilst this is not new, the practice of engaging in beneficiaries’ own spaces is often undervalued. Twersky, Buchanan and Threlfall (2013) note:
In bypassing the beneficiary as a source of information and experience, we deprive ourselves of insights into how we might do better – insights that are uniquely grounded in the day-to-day experiences of the very people the programs are created for.
They suggest that unlike the private sector and their customers, in the development sector, it is easier to ignore the beneficiaries who often ‘express gratitude for even subpar effort’ (ibid.).
Integrated programmes such as the MVP are intended to address problems in a holistic manner and, by inference, acknowledge that people do not experience life through a series of sectoral lenses (such as separately for health or education). However, if such projects fail to consider people’s experience and their value given to change, they merely co-ordinate different sector-based interventions with the ‘hope’ that there is some synergistic benefit; rather than recognise the importance of sequencing interventions, moving at the pace that people want, and adapting to new emerging possibilities and challenges. Each intervention within an integrated approach creates a series of ripple effects which need to be adjusted to. Immersion studies can help identify these effects early because of the opportunities provided, by combining listening with observation, and experience within beneficiaries’ ‘own space’, with the intrinsic advantage over ‘invited spaces’.
The use of immersion research described in this article may help address some of the criticisms of participatory practice levelled by Cooke and Kothari (2001) as noted in the introduction. The informality and ‘hanging out’ in beneficiaries’ ‘own space’ has the potential to address many of the issues raised about co-option, unhelpful use of power, outside agendas, and problems encountered by interaction only in ‘invited spaces’. Furthermore, it derives value from the researchers’ shared experience and opportunities for observation, neither of which are emphasised in other qualitative methods. Conventional ethnography is often limited to a single researcher spending extended periods of theory-led immersion research in the community. There are benefits of having many researchers undertaking immersions concurrently, and without theory or evaluation framework lenses. Short immersion time and scope can arguably be trade-offs for insights which may be hard to gather from ‘invited spaces’. Immersion research provides an informal means of engagement which addresses much of the concerns about the disincentives and costs to participate. Where empowering forms of participation are not undertaken because of poor participation practice, because of the required independence of external evaluation, or because people actively eschew public participation, immersion research may indeed provide a middle ground or ‘passive participation space’.
There is a well-known African proverb, ‘Only the one sitting on the anthill knows the ants are biting’. When researchers immerse in villages and live with families, they can at least get some taste of the ants biting and some insights on emic perspectives.
Notes
* This issue of the IDS Bulletin was prepared as part of the impact evaluation of the Millennium Villages Project in northern Ghana, 2012–17, funded by the UK Department for International Development (DFID) (www.dfid.gov.uk). The evaluation was carried out by Itad (www.itad.com) in partnership with IDS (www.ids.ac.uk) and PDA-Ghana (www.pdaghana.com). The contents are the responsibility of the evaluation team and named authors, and do not necessarily reflect the views of DFID or the UK Government.
1 Technical Advisor, Empatika and External Research Associate, School of Development Studies, University of East Anglia.
3 Participatory Development Associates Ltd, Accra, Ghana.
4 External evaluation is defined for the purpose of this article as an independent function, reporting separately to the funder to provide rigorous, impartial evidence primarily for accountability purposes.
5 www.reality-check-approach.com.
6 Also reducing ‘sponsor bias’.
7 While short compared to many ethnographic studies, these are longer periods of interaction than focus groups, interviews, and participatory approaches such as PRA.
8 Including categories derived through PRA wellbeing rankings.
14 Tuo zaafi, a cooked very thick porridge of maize and water.
16 http://millenniumvillages.org/the-villages/.
References
Carr, E.R. (2008) ‘The Millennium Village Project and African Development: Problems and Potentials’, Progress in Development Studies 8.4: 333–44
Cooke, B. and Kothari, U. (eds) (2001) Participation: The New Tyranny?, London and New York NY: Zed Books
Cornwall, A. (2004) ‘Introduction: New Democratic Spaces? The Politics and Dynamics of Institutionalised Participation’, IDS Bulletin 35.2: 1–10 (accessed 14 September 2018)
Gaventa, J. (2006) ‘Finding the Spaces for Change: A Power Analysis’, IDS Bulletin 37.6: 23–33 (accessed 14 September 2018)
Geertz, C. (1998) ‘Deep Hanging Out’, The New York Review of Books 45.16: 69–72
Hickey, S. and Mohan, G. (eds) (2004) Participation: From Tyranny to Transformation? Exploring New Approaches to Participation in Development, New York NY: Zed Books
Mahmud, S. (2007) ‘Spaces for Participation in Health Systems in Rural Bangladesh: The Experience of Stakeholder Community Groups’, in A. Cornwall and V.S. Coelho (eds), Spaces for Change? The Politics of Citizen Participation in New Democratic Arenas, London and New York NY: Zed Books
Twersky, F.; Buchanan, P. and Threlfall, V. (2013) ‘Listening to Those Who Matter Most, the Beneficiaries’, Stanford Social Innovation Review (accessed 7 June 2018)
White, H. (2009) Theory-Based Impact Evaluation: Principles and Practice, 3ie Working Paper 3, New Delhi: International Initiative for Impact Evaluation (3ie)
© 2018 The Authors. IDS Bulletin © Institute of Development Studies | DOI: 10.19088/1968-2018.163
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non Commercial No Derivatives 4.0 International licence (CC BY-NC-ND), which permits use and distribution in any medium, provided the original authors and source are credited, the work is not used for commercial purposes, and no modifications or adaptations are made. https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode
The IDS Bulletin is published by Institute of Development Studies, Library Road, Brighton BN1 9RE, UK. This article is part of IDS Bulletin Vol. 49 No. 4 September 2018 ‘The Millennium Villages: Lessons on Evaluating Integrated Rural Development’; the Introduction is also recommended reading.