Now that you have designed your tools and collected your data, it’s time to analyze it. This section will explain how to deal with both quantitative (numbers, for example, counts, frequencies, or averages) and qualitative (words, such as responses to open-ended survey questions, interviews, and focus groups) data.
At this point, your evaluation plan has hopefully set you up so that you’ve only collected data that will be helpful in answering your evaluation question. If you do get lost in data, you can always use your evaluation question to as a guide, and ask if the ways you are analyzing or summarizing the data are getting you closer to answering the question.
Quantitative data analysis can be intimidating if you do not have training in research or statistics. Most evaluation data you will collect, however, can tell you a great deal just by looking at the frequency distributions of answers or the mean (average) score from a scale. You do not need statistical software to analyze quantitative data. Both Excel and the online survey service you use can do basic counts, percentages, and means, as well as generate graphics such as pie and bar charts.
Analyzing Numbers and Counts in Excel
Some of the questions you ask in a survey are likely to be yes/no, a scale of agree to disagree (or another type of scale), or a list of options. These types of questions can best be analyzed by counting up the number of respondents who answered each of the options and calculating percentages.. If you enter the survey questions into a spreadsheet, with one line per respondent and one column representing the answer to a question, it is easy to count up the responses.
If you decide to use graphs, charts or other visuals, be sure to keep them simple so readers can understand them without a lot of extra explanation.
- Check out the Data Viz Checklist for a guide on how to keep your graphs clear and easy to understand.
- The Evaluation Toolkit from the Pell Institute has a great step-by-step description of how to analyze quantitative data.
Data Analysis in Online Survey Sites
Online survey programs like SurveyMonkey or Google Forms are very useful, not only to collect data, but also to analyze it. Even if it makes more sense to collect data through paper surveys, you can enter the responses you receive into the online survey program and run a summary report.
This can take away the intimidation factor of analyzing quantitative data and give you more options for how to summarize and compare different questions than you might easily be able to do in Excel.
Qualitative data can be overwhelming because of the sheer volume that can be collected in each interview. It can also be challenging to figure out how to make sense of it. Too often, qualitative data is simply used to pepper a report with quotes to demonstrate a point the writer is trying to make. Not only does this underuse the effort your team and the participants put into the evaluation, it also underestimates what you could have learned by doing a more systematic review of the data.
Coding Qualitative Data
To analyze text that you collect, whether through interviews, focus groups, or surveys, you should begin by reviewing your notes or the answers to a survey. Your objective is to identify key themes. If you have gathered a lot of qualitative data, you will want to use tools that will help you sort through it.
The first step in qualitative analysis is to reduce the total amount of data to something easier to use! This step is called “coding” because you assign short phrases (codes) to longer chunks of data that highlight the key point being made. For example, if an interviewee commented, “Medicaid expansion is important, but it’s not realistic in this political environment. Honestly, I don’t think it ever will be, which isn’t really a statement about whether it matters, just whether we should spend any energy focused on it” you may want to assign the code, “Medicaid expansion important, but politically unrealistic.” And just like that, you go from 46 words to six.
You can also reuse codes – applying the same short phrase to different interviewees’’ statements or different statements within an interview to help you see where commonalities can be found.
To do the coding, you can highlight passages and use comments in the margins for the code label/theme if your notes or transcript are in a word processing file. If you have organized information in Excel or other spreadsheet programs, you can create the codes in the spreadsheet in a second column next to each interview.
Analyzing Qualitative Data
Once you have your data coded, you have multiple options to find patterns and unexpected outcomes.
- You can sort and reassemble your data by the codes, thereby discovering if there are certain perspectives more likely to be articulated by one type of stakeholder rather than another (e.g. policymakers rather than grassroots advocates).
- You can reassemble the data sorting the spreadsheet by categories, which are a set of codes/themes that have been grouped together, for example grouping all the codes that relate to opportunities for policy successes.
- You can show relationships among categories or themes.
- You can look at the relative weight given to certain types of themes (shown in percentages of comments that were coded with that theme) overall or by subgroups, which may show, for example, that certain groups talk frequently about one issue while others rarely mentioned it.
There are some low-cost online programs that make analyzing qualitative data easier, once you learn the program!
- Dedoose is an affordable, easy-to-use online platform for coding qualitative data. The codes can be exported to Excel, and you can easily share coding projects among several people for collaborative coding.
- QDA Miner Lite is a free qualitative analysis application (there is also a paid version with more features) that allows you to code not only text data, but also visual information such as drawing, photographs, and paintings.
- Weft QDA is a free, open-source software application for smaller qualitative coding projects.
- The Evaluation Toolkit from the Pell Institute has a great step-by-step description of how to analyze qualitative data.