On Monday, Rosie and I went to an evaluation surgery organised by the Royal Academy of Engineering (RAENG) in Bristol. It was an afternoon workshop lead by Ben Gammon, which purpose was to help us evaluate projects awarded with Ingenious Grants, by completing an evaluation plan. Ben shared with us his 5 key points in evaluation:
- What do you need to find out?
- Is data going to be useful? Will it actually help you to change something?
- Consider the time and money that doing getting that data would take
- Choose an effective method to gather the data
- How are you going to disseminate the data?
He also gave us very great advice on does and don’ts when designing a survey. As we are survey fanatics we took great attention to these. Here are some of them:
- Always avoid “and” in any survey questions, as you would be asking two different things in the same question, and you’ll never know if people are responding to one, the other, or both of them.
- Never use “Why” in a question, it will give you too broad answers, which are difficult to analyse.
- When using a rating scale, include the same number of positive and negative scale points and do not group different points together in your report.
- Always have somebody else to check your survey before you send it out.
We also talked about how we can encourage people to fill in our surveys, while collecting the most honest data possible. We learnt that giving a small incentive to everyone responding the survey might work better than offering the opportunity of winning something bigger. As for honesty, keeping the survey anonymous and setting an example by letting the audience now why we need the information and how long it takes to complete the survey is always a nice start. One of the last points discussed was the dissemination of our results, which should ideally be summarised in a list of key findings followed by the key recommendations deduced from them. Ben decided to end the workshop with his TAP rule for evaluation:
- Topic: define the main subject of your survey very clearly.
- Applicable: ask only the questions which are relevant to your project
- Perspectives: make sure respondents understand the perspective they need to answer from (whether they need to respond as an individual, member of a body, etc)
After the workshop, we went out for some networking under Bristol’s rainy spring sky. It is now time for us to plan the evaluation of “I’m an Engineer, Get me out of here”, we’ll keep you update on this.
