How do you know what you need to paint a clear and compelling picture? There is a great deal of existing data out there, and much of this information can be collected so the first step is to focus on the purpose of using data by asking questions such as:
- What upcoming decisions would be more strategic if you had data to inform them?
- What questions are being asked that relate to that decision?
- What types of data would help answer the questions?
Answering these questions can help in selecting actionable data to support making decisions that can lead to better outcomes and a greater change in the world as a result of your work. Determining your focus can help you strategically gather data to meet your needs. There are many different types/sources of data, including but not limited to:
- Evaluation – What are the cause and effect relationships between strategy and outcome?
- Assessment – What is the current state?
- Forecasting – What is possible in the future?
- Retrospective – What are the range of options that have been tried in similar settings or in the past in this setting?
- Perspectives – How is the problem/solution viewed differently by different actors?
- Drivers – What is causing the problem?
- Context – What else is influencing the problem?
The first step in any process when using data, extensive or not, is to be clear about the decision-making opportunity that data can inform, which means clearly understanding three things:
- The decision;
- The decision-makers; and
- The type of decision being made.
Decisions can range from high-level strategic decisions about how a problem is being defined and the solution being selected to tactical decisions made during implementation in order to steadily improve the outcomes of the work. Different types of decisions suggest different data needs so being very clear about what decisions you hope to inform through the use of data is a critical step.
In addition, there are many different types of potential decision-makers:
- Designers/strategy planners: The leaders responsible for understanding a problem and finding a solution;
- Implementers: The leaders and staff who implement, adapt, and improve solutions that are being applied to the problem;
- Board members/funders: The organizations and individuals that have decision-making authority over the deployment of resources needed to solve a problem; and
- Other audiences who have an invested interest in seeing the problem is solved as well as a willingness to take action themselves to help solve it.
For each decision you hope to inform with data, think about the type of decision being made. It will help in exploring the type of data you will need. Decision types include factors such as:
- Defining the problem;
- Understanding the context;
- Identifying potential solutions;
- Selecting a solution to test;
- Developing an implementation plan;
- Revising/refining implementation along the way;
- Assessing appropriateness of the solution along the way;
- Identifying whether resources should continue to be invested in the solution; and
- Identifying whether to get involved in trying to help solve the problem.
Learning questions are an important way of ensuring the data you collect will be useful. By taking the time to carefully craft the question you hope to answer, you are giving yourself direction on what information will be useful and what information is interesting, but not directly related. Think carefully about the decision that the data is intended to inform. What questions are the decision-makers asking or will they be asking as they approach this decision?
Learning questions often start with:
- What is the context of…?
- How did our…?
- What happened when…?
- What influenced…?
- What changes were implemented…?
- What patterns have we seen…?
You are likely to have multiple questions that you can use data to answer; therefore, it is important to prioritize which questions to answer and at what depth. You must consider the extent to which answering the question can advance the work, whether it relates to a major threat or opportunity, and the timing during which stakeholders might use the results to inform a decision.
Some questions will require much deeper investigation than others in order to answer. If a lower priority question can only be answered with in-depth data gathering, chances are it is not worth your time. If it requires fairly minimal data to answer, it may be worth your time, but keep in mind how often you are bringing new information to stakeholders and asking them to delay their work so they can consider it. Prioritizing is not just about your time, it’s also about the time it takes your partners to participate in the learning process.
With your question in hand, the next step is to consider the types of data that can help you answer the question. Let’s explore a few examples:
Example 1: Who is most affected by the problem and why?
This type of question suggests a need for data that defines the breadth of the problem. Look for demographic or surveillance data. The relevant public agencies will often have data that can help you understand who is most affected by a problem. However, this high level quantitative data might not necessarily help with figuring out the “why” of the problem. You may find that learning from service providers who work on the issue every day, consumers/community members affected by the problem, or other researchers who have already investigated the problem can help you far more when answering the “why” part of your question.
Example 2: Which solution is most likely to solve the problem?
This type of question presumes there are solutions you are exploring, which suggests data that related to how similar solutions have played out in other communities could be useful. Looking for reports and publications and talking to experts can be helpful. It’s also important to recognize that although the problem you’re trying to solve is most likely not unique, it will have challenges and nuances unique to your context. Collecting the insights of community members, providers, leaders, and others who understand the local context and can assess the potential challenges and benefits of different solutions can bring new voices into the collaborative dialogue where decisions are being made.
Example 3: To what extent and in what ways is the solution helping to solve the problem?
Of the examples, this type of causal relationship question is the least likely to be answered through existing data sources. While surveillance data might tell you whether a problem is decreasing, it often takes years before the effectiveness of a solution can be seen at the level of community-wide data. In contrast to the two earlier examples, this is an evaluation question and suggests engaging in an evaluation of the effectiveness and appropriateness of the specific policy and practice changes that are part of the overall solution.
Applying an equity lens can be a part of all aspects of your data process. What you collect, how you analyze and disaggregate the data, and how you present the data all affect how decisions are made. The right data allows for a focus on eliminating disparities and supporting solutions so all people are allowed the opportunity to reach their full potential. Inequities exist due to a myriad of systemic social, economic or environmental factors, which should constantly be brought forth through the data processes. When you make decisions about what data is important, you are either creating the opportunity to surface and highlight inequities or hide inequities by using data that cannot be parsed in such a way as to explore differences by groups, communities or neighborhoods.
By applying an equity lens to data gathering and dissemination, you can help identify, assess, and reflect on the potential impact a solution has, or the impact a solution already underway is having, on communities with different needs within their unique social context. It creates the opportunity to mitigate negative impacts, enhance positive impacts, or prioritize solutions for populations most in need.
To apply an equity lens, ask about which people, geographic areas, or other groups are most affected by the problem historically, what social, economic and environmental data could also be overlaid, and where resources are currently dedicated. All of this should lead to prioritizing your approach, practices, and decisions. Creating the urgency and direction through a clear, data-centered approach, identifying the disparities, and highlighting the right data will help allocate the right resources to the people and places that need it most.
It is important to consider when you plan to use the information and designing your process accordingly. Your timeframe will shape what types of data you can gather, how quickly you need to move from gathering to analyzing, and what decisions you can inform. Do you have a short timeframe during which you can use the information to improve your strategies? If so, you want to ensure your data gathering and analysis can realistically happen within that timeframe. Is your timeframe longer? You may be able to go deeper in your data collection or do more in-depth analysis.
Similarly, before identifying data strategies, you want to assess your capacity to undertake data gathering, analysis and reporting. Consider questions such as:
- Do you have a staff with the skills and experiences needed for basic quantitative analysis such as identifying means, percentages, or otherwise manipulating numbers in Excel or another program?
- Has anyone on your team worked with qualitative data in the past? If not, is anyone ready to build this skill?
- If your team does not have the skills or time to interpret data, do you need to find an external analyst?
Once you have selected your questions, identified the data to collect, and are comfortable with your timeframe and scope, it is time to start collecting your data. There are myriad sources of data from which you can draw. When beginning your data collection process, it can be helpful to keep in mind the 4 Rs.
Data should be:
- Recent – although there are often lags in data, use the most recent available.
- Relevant – collect data that is relevant to your geographic scope and to the problem you and your partners are addressing.
- Reputable – if you are using data external to your organization, source data from reputable sources such as federal or state databases or respected organizations.
- Responsible – consider whether the data allows you to address questions of equity.
This toolkit is focused on how to select the right data to inform your decisions and the process of using the data. It is not designed to help you collect high quality data and analyze it in effective ways. Don’t worry though – there are great resources available to guide both critical steps.
Spark’s other toolkits explore some of the more innovative data collection and analysis strategies, which may be of use when answering questions related to complex, dynamic problems and solutions. Look at the methods pages in:
Beyond Spark’s resources, there are high quality, accessible toolkits to guide you through data collection and analysis such as:
- The Community Tool Box, a free online resource to help develop stronger communities, has a great section on collecting and analyzing data for social change initiatives.
- The Evaluation Toolkit from the Pell Institute has a great step-by-step description of how to analyze qualitative data and quantitative data.