Evaluation Toolkit – Choosing the right tool

Once you have decided exactly what you want to find out and taken account of the facilities and possible limitations available, it is time to choose the most appropriate evaluation tool.

Think before you choose

Until you have answered all the questions in the chapter on planning your evaluation tool, it is easy to unintentionally select a data collection
method that does not deliver what you want.

For example, you might go to all the trouble of compiling and distributing a questionnaire, only to find that the results don’t tell you very much, and it may have been better to use an alternative such as Tool 6 – Snapshot Interviews to get more specific, focused feedback. Worse still, your audience may have got bored or even frustrated by filling out “yet another” questionnaire, thereby detracting from their experience of an otherwise successful event. Once you’ve answered the questions above, you’re in a better position to decide what methods will be best for your activity or project.

Pathways to choosing the right tool

To help you select which tool is most appropriate for your planned outreach activity we have put together a set of interactive tables comparing their features.

Depending on your interests, you may find various pathways helpful. Possible selection criteria that are included in the interactive tables are:

• Suitable audience(s) e.g. primary or secondary schools, interested adults or general public
• Appropriate activity type(s) e.g. lecture presentation, ongoing series,
  drop-in workshop or online
• Response type e.g. online, written, verbal and
  whether it is short or long
• Time required to prepare the evaluation methods, implement the evaluation methods and analyse the data collected
• Type of information gained e.g. what works / suggested improvements / evidence of changes
• Most relevant Generic Learning Outcomes
• When they might best be used e.g. during, beginning/end, or after an event.

All the tools have an “At a glance…” summary table with icons showing which criteria they are most suited to.

Tailoring and testing your chosen tool

Once you have chosen the tool you think is most appropriate for your purpose, you may need to make some adjustments depending on, for example, your audience or available resources.

Aim to integrate the evaluation into the activity as much as possible, rather than have it as a separate “add-on”. That way, participants feel that it’s a natural thing to be part of, rather than something they have to do after the activity is over and they want to leave.

Make sure the language used in the tools is appropriate to the reading level of your audience. For instance, for primary school children, sentences need to be kept short and simple. In fact, it’s always better to err on the side of simplicity – adult audiences appreciate this too and it makes the evaluation quicker. You may want to consider using a reading age calculator (e.g. http://datayze.com/readability-analyzer.php) to test how complicated your text is.

If people are enjoying doing the evaluation, they are more likely to take their time and provide meaningful responses, so make it fun, for the
participants and for you! For this reason we’ve tried to include a number of engaging tools within this Toolkit. Similarly, if the evaluation takes too long (or even looks too long), people simply won’t do it, so keep the instruments short – this is not a school exam!

It is also a good idea to pilot your questions and instruments in advance – it’s worth the time it will take. Have three or four people who are similar
to your intended audience (e.g. primary school students, members of the general public) try your tool to make sure they understand what is being asked. These people can be family members or friends but should NOT be colleagues or others with similar scientific backgrounds.

Finally, do consider using multiple tools to gain different perspectives, or even as complementary stages in the process. For example, using Tool 9 – 3 Words at a pilot event with a smaller group might help you identify what questions to include in a pre-post quiz (perhaps using Tool 3 – Mentimeter or similar) for later events with larger audience sizes..

Back to Evaluation Toolkit homepage