Developmental evaluation was designed for settings where innovation and adaptation are an ongoing part of the strategy and where the environment and the potential solutions to the problem are complex.
The term was originally coined by Michael Quinn Patton and has been used by evaluators working in a wide array of arenas, from small community-based projects and programs to large multi-national initiatives.
The overall purpose and theory behind DE has been defined in depth by Michael Quinn Patton and explored in a variety of other resources, including guides on the process and skills of developmental evaluation. To learn more about the theory and concepts of developmental evaluation, please consult the resources below.
- Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use, by Michael Quinn Patton. This is an introduction to DE, including an illustration of how it can be used in a wide range of settings for many different purposes.
- A Developmental Evaluation Primer, by Jamie Gamble, published by the J.W. McConnell Family Foundation. This primer introduces the concept of DE and provides tools to support its use.
- DE 201: A Practitioner’s Guide to Developmental Evaluation, by Elizabeth Dozois, Marc Langlois, and Natasha Blanchet-Cohen,published by the J.W. McConnell Family Foundation. This guide builds on the concepts outlined in A Developmental Evaluation Primer by articulating some of the key practices associated with this work. These practices were identified and developed through a three-year multi-site case study involving developmental evaluators at six sites across the country.
Developmental evaluation is a tool for social innovators to develop social change initiatives in complex and uncertain contexts. It is specifically intended for situations where innovation, radical change from past practices, ongoing development, or complexity is present.
- Unlike formative evaluation, DE’s purpose is to help people develop a way to move forward when the future is uncertain or unknown; it is not intended to refine and adjust an expected course of action.
- Unlike summative evaluation, DE does not seek to test the efficacy of a predefined program or practice.
Developmental evaluation has a very different starting place from summative evaluation. You don’t need to know the road ahead before an evaluator can help you. Rather, DE can help you explore the road as you go down it, making it a powerful tool amidst uncertainty.
- To determine whether developmental evaluation is right for your setting, see the Developmental Evaluation Diagnostic Checklist by Mark Cabaj, published by Tamarack. This tool will help you examine the complexity of the problem, uncertainty around solutions, and organizational culture to determine whether DE is appropriate.
Developmental evaluation is designed to for environments with a high level of complexity. For a developmental evaluator, it is critical to understand how complexity functions. Concepts of complexity can deeply inform many of the evaluation questions you craft and the frameworks and methods you use.
Complex environments include:
- Social embeddedness: Many different feedback loops, where everyone is both shaped by and shaping the social networks that surround them and influence their behaviors. This includes the influences of language, knowledge, moral and legal systems, culture, and more.
- Emergent properties: When the whole cannot be understood by looking at the parts. Complex environments are like the brain – you cannot understand it piece by piece, nor could you reconstruct it if you took all the pieces apart and attempted to reassemble them.
- Causality is non-linear: The interactions and influences from the broader environment have an impact on the outcomes. In other words – the program or service is not the only reason that outcomes occur.
- Lack of central control and a high level of spontaneity: Change can and often does originate from many different directions. It is unexpected and frequent.
- For a very accessible description of complexity theory, visit Tamarak’s page on complexity.
Developmental evaluation also elucidates how the issues at hand are embedded in systems and how systems can be part of the solution. Developmental evaluators need to understand the concepts of systems and be able to observe, document, diagram, and discuss systems, as they are often the frame through which problems and solutions can be understood in a complex environment.
- Many different perspectives, each of which may view the problem and solution in very different ways.
- Inter-relationships, where each perspective within the system is connected to and influenced by other perspectives.
- Boundaries, where systems are best understood by defining what is within and outside of them.
- For a very accessible description of systems, visit Cornell University’s Office for Research on Evaluation.
- GEO’s Systems Grantmaking Resource Guide allows you to look through a wide variety of systems tools and resources to put a customized systems approach into practice.
Developmental evaluation requires many competencies beyond research skills and processes. Sometimes called “soft skills”, these competencies cross a range of process and perceptive skills. This toolkit does not address these skills, but rather provides the concrete practices that can be undertaken with such skills.
- For more on the process and perceptive competencies, consider reading The Art of the Nudge: Five Practices for Developmental Evaluators, by Marc Langlois, Natasha Blanchet-Cohen and Tanya Beer, published by the Center for Evaluation Innovation.
Not all settings are equally ready for developmental evaluation. In an environment where failure is not an option, developmental evaluation may not be a good choice. The issue of readiness for developmental evaluation has been explored in multiple resources, including discussions in two papers and a readiness checklist that we use at Spark.
The protocol below provides questions that may be useful when exploring a new DE setting. The questions not only help to identify readiness level, but will also facilitate planning the evaluation in a manner that respects higher and lower levels of readiness.
- Check out the readiness section in DE 201: A Practitioner’s Guide to Developmental Evaluation by Elizabeth Dozois, Marc Langlois, and Natasha Blanchet-Cohen, published by the J.W. McConnell Family Foundation.
- To explore readiness for learning more broadly, see Evaluation for Strategic Learning: Assessing Readiness and Results by Anna Williams, published by the Center for Evaluation Innovation.
- The Developmental Evaluation Diagnostic Checklist by Mark Cabaj, published by Tamarack is a resource to help you determine readiness for DE . The tool will help you examine the complexity of the problem, uncertainty around solutions, and organizational culture to determine whether DE is appropriate.