Evaluation Toolkit – Analysing your data

Analysing your data

Once you’ve collected your data using the tools, you’ll need to analyse it. Here, we describe how to prepare qualitative data (from open-ended questions) for analysis and how to use two common analysis techniques for qualitative data. We also show you how to carry out a basic analysis of closed-response data.

Preparing Qualitative Data for Analysis

Before we analyse our data, we need to make sure there are no obvious errors. These could include repeated entries, typos, misspellings or similar. The following instructions use samples from a hypothetical dataset of teacher feedback (collected using Tool 9 – 3 Words) on an imaginary website, Europlanet for Schools [1].

For example, the raw data from our teacher feedback might look like Table 1.

What three words would you use to describe the Europlanet for Schools website?

Raw dataset

123
relevantChallengeuseful
navigableUsefulcomprehensive
excellentDisappointinginformative
Usefulinformativerelevent
usefuleasy to navigateextensive
challengingtoo scientificaccessible
informativeMotivatinggreat activities
accesibleUsefuleducational
excellentinformativecan be frustrating to search
variedUsefuldisappointing
funEngaginginspiring
navigatableHelpfulinformative
easy to usePracticalhard to search
innovativeInterestingclear
comprehensiveExcitingcomplicated
usefulInformativeeasy to use
technicalInformativeuseful
inspiringChallengingexciting
relevantChallengeuseful

We can see that there are a few minor issues with the data, for example:

  • There are some small spelling mistakes e.g. relevent / relevant
  • Some words are capitalised and some not e.g. Useful / useful
  • Linked but slightly different words have been used e.g. challenge/ challenging (these are known as “stem words” in language analysis)
  • Some people have entered a short phrase instead of a single word in some entries (e.g. “easy to use”). When creating word clouds, the software thinks of this as three separate words (“easy”, “to” and “use”) so we need to force it to recognise those terms as being connected.
  • Some phrases / words have the same meaning but have used different words e.g. navigable / easy to navigate / good layout / navigatable

These differences will artificially reduce the frequency of such terms by splitting them over separate entries.

Step 1: Remove Duplicates

The easiest way to remove duplicates caused by capitalisation is to copy & paste the text into MSWord (keeping it in the table format) and then use the ‘case’ button to toggle all the text to ‘lowercase’.

You can then paste the table back into Excel.

Step 2: Connect Phrases

Highlight the text, then select Edit > Find > Replace and replace all spaces (“ “) with a tilde (“~”) sign (alt-n on a Mac) – so “easy to use” becomes
“easy~to~use”. This tells the word cloud software that those terms are now linked, but displays them as spaces for easy reading in the final word cloud.

Step 3: Standardise Words and Phrases

To check our data for further errors it is easier to look at the whole list of words it contains. If we input data to the www.wordclouds.com site, there
is an option to select ‘word list’ – clicking on this brings up a list of all of the words contained within the data file, as well as their frequencies (how often they occurred within the text). These can then be arranged into alphabetical order (this helps us to spot issues with the data more easily). The word list can then be copied back to the spreadsheet and spelling mistakes can then be easily corrected and linked words and similar phrases standardised, as shown in Table 2. The “Edited Word” column notes all the places where the original word needed editing.

The “Updated List” shows the same data with all the edits applied – so, for example, 1 count of “accesible” and 1 of “accessible” have been combined
into a single entry “accessible” with a frequency of 2.

FrequencyWordEdited wordFrequencyWord
1accesibleaccessible2accessible
1accessible3challenging
1can~be~frustrating~to~searchhard~to~search1clear
1challengechallenging1complicated
2challenging2comprehensive
1clear2disappointing
1complicated2easy~to~use
2comprehensive1educational
2disappointing1engaging
1easy~to~navigatenavigable2excellent
2easy~to~use2exciting
1educational1extensive
1engaging1fun
2excellent1great~activities
2exciting2hard~to~search
1extensive1helpful
1fun8informative
1great~activities1innovative
1hard~to~search2inspiring
1helpful2interesting
8informative1motivating
1innovative3navigable
2inspiring1practical
2interesting2relevant
1motivating1technical
1navigable1too~scientific
1navigatablenavigable9useful
1practical1varied
1relevant 
1releventrelevant
1technical
1too~scientific
9useful
1varied

When all errors have been corrected, we are ready to start analysing the data with our analysis tools, Word Clouds and Thematic Coding.

To find out more about analysing closed questions, see Tool 11 – Post event surveys and Tool 7 – Pre / Post Quizzes.

Back to Evaluation Toolkit homepage