Category Archives: Evaluation

Reflections on SME funding

As part of our funding agreements in recent years we’ve been trying to diversify our funding streams for I’m an Engineer and I’m a Scientist. Inevitably one suggestion for additional funding has been to approach companies employing engineers and scientists.

The obvious first step was to approach those companies known to support STEM engagement. BP, Rolls-Royce, BAe, GSK, Pfizer, Thales etc. It wasn’t simple. Finding the right people at the right time wasn’t simple. Creating a proposition that worked for them wasn’t simple.

It was put to us that these companies are constantly being asked to support projects and that we may have more success with SMEs.

So we picked a few upcoming zones (Health, Space, Production and Energy) and set a researcher a task of finding 100 SME’s in each field where the CEO had given their contact details on their website. We figured that if they did that they were more likely to be interested in engagement and online engagement in particular.

We sent a letter and some flyers to each CEO asking them to encourage engineers to sign up. The response rate was not spectacular at about 5 engineer signups per 200 letters sent.

The quality however was good. The engineers participated fully and had a variety of experience. But participation was only part of the conversation. We wanted to turn participation into funding. Next step was to talk to a couple of decision makers in a couple of the companies that had provided engineers.

Both were CEO’s who were very, very supportive of STEM education and outreach. They give up their own time for it and encourage their employees to do the same. Both had companies which relied on brilliant cutting-edge engineering to be a success. They were both research intensive companies in a start-up phase. But being a start-up meant that there was no spare cash in the company and investors wouldn’t allow discretionary spend on activities like outreach.

And there lies the rub.

The companies that rely most on cutting-edge research and be supportive of outreach activity are more likely to be unable to fund outreach activities because of unprofitable stage of their corporate lifecycle. It’s only once they become less research intensive that they are able to afford to financially support these types of activities.

Obviously we’re dealing with a tiny sample and the support shown by them means we will continue to approach SME’s for engineers and continue to sound out funding opportunities. But it isn’t going to be a simple solution for funding diversification in the near future.


Posted on January 13, 2016 ModShane in Evaluation | Leave a comment

How does I’m an Engineer affect students understanding of engineering?

We know that I’m an Engineer gets students interested in engineering  and inspired to become engineers themselves.  However, one of the key objectives of the project is to smash engineering stereotypes; that the students learn that engineering is not just about car mechanism, spanners and hard hats.

Measuring the students’ change in understanding of engineering has been difficult: closed questions tend not to work when assessing this, as they can be very leading. We decided to ask students to evaluate their own learning. Of course, this has its own limitations, but so far it is the best method we have found.
We asked students for their degree of agreement on whether since taking part in I’m an Engineer, they know more about different aspects of being an engineer.

Engineer perception change - student survey results

Responses from 131 students told us that taking part in I’m an Engineer helps them to improve their understanding of what engineering is and what engineers do.  Almost all the students who filled in the survey said they knew more about the type of tasks that engineers do, and the type of projects they work on. And only around 10% of students didn’t think they knew more about the skills required to be an engineer, the type of people who work as engineers, and engineers’ role in society.

Students show their interest about these points during the event, when they ask questions like “How is your project going to make an impact in society?” , “What skills are involved with your specific area of engineering?” or “Do you fell your career is repetitive at times, or are you given different tasks?”

Thank you so much for your advice it has really has helped me understand engineering more knowing the use that science and maths can give to engineering” – putptre06, student

a totally worthwhile experience with all 5 of my classes, they thoroughly enjoyed it but more importantly gained a realistic insight into the life of an engineer.” – Teacher

Posted on July 28, 2015 modangela in Evaluation, News | Leave a comment

How does I’m an Engineer affect students’ attitudes to engineering?

We know I’m an Engineer has a positive effect on students. Teachers tell us their classes “got fired up about engineering”, some students “chipped a nail typing to the engineers” and others admit they “didn’t even know that was a job, how cool!!”

Anecdotes are fine, but as evaluation geeks we want numbers too.

We asked students if they think engineers had an interesting job, and whether they’d like to work as engineers in the future, before and after taking part in I’m an Engineer. We gave them a choice of 5 options – from the very positive to the very negative, with a middle neutral one – to reply. 76 students filled in both the pre-event and post-event surveys.

Students have a good perception of engineering, and I’m an Engineer improves it

The majority of the students thought engineers had a “very” or “fairly interesting” job, and none of them thought it was “definitely not interesting” before taking part. This got even better after I’m an Engineer: the percentage of students who thought engineering jobs were “very interesting” went from 37% pre-event to 54% post event.

I’m an Engineer also helped students make up their minds: the 11% of students who “didn’t really know” if engineers had an interesting job before taking part plummeted to 1% after the activity.

Engineers interesting Job 3- student survey J14

My pupils were buzzing after the event, they really enjoyed it. They were even arguing about which engineer should win, so the engineers involved have made a huge impression on them.” – Jacqui Foord, teacher June 14

Students get inspired to become engineers.

The percentage of students who wanted to become engineers jumped from 35% to 47%, and the percentage of students who were sure they would like to be an engineer doubled after taking part. On the other hand, the percentage of students not interested in working as engineers decreased from 35% to 21% after I’m an Engineer.

would like to be eng 3 - student survey J14

I’ve done a follow up of giving them a week to “constructively daydream” and come up with an idea or invention – even if it would be impossible to produce it yet. I’m really impressed by their ideas – plus they’ve signed contracts giving me 10% of future income from their inventions. Dragons Den, here we come!!” – teacher, March 13

And there is yet more evaluation to come. We are looking into how taking part in I’m an Engineer changes students’ understanding of what engineering is, and what engineers do. We are gathering data about that just now, from students taking part in the June 2015 event. We will be back with more graphs and quotes soon!

Please leave a comment to say “hi” or ask for any clarification. The main purpose of this post is to share our learning and start a dialogue.

Posted on June 29, 2015 modangela in Evaluation, News | Leave a comment

Evaluating I’m a Engineer

In March 2012 we ran I’m an Engineer for the first time. We’ve had engineers take part in the sister I’m a Scientist event so running a version specifically for engineers was an obvious move.

Before the event started we had some unknowns, despite considering ourselves well versed in running online engagement events. Would engineers want to take part? Would we be able to recruit Maths and D&T teachers? What questions would students ask?

When it came to evaluation time these were some of the key questions to answer. We’ve collected feedback from teachers, students and engineers. We’ve used their responses and data from the site to evaluate the event, and have written up our findings in the report below.

I’m an Engineer evaluation report

Here’s a summary of what we found:

Did engineers want to take part?

Yes. By the end of the first three months we had over 165 engineers apply for the 30 places. The mix was good too. 34% were female, 50% said they had been working for just 10 years or less, and 2/3rds worked in industry. The last figure differentiates them from our scientists of whom 2/3rds work in academia – see page 18 for more details.

Could we recruit Maths and D&T teachers?

Recruiting Maths and D&T teachers wasn’t so easy. We worked with DATA and NCETM. We had a good response from Maths teachers, but D&T teachers were more difficult to reach – see page 25.

What questions did students ask?

In I’m an Engineer we found that students were more interested in engineering as a job than as a discipline – see page 11. Over 50% of questions were about careers and workdays. 22% were specifically about some kind of engineering, often about the zone theme or the engineers’ work. Other popular questions were more personal ones. Students can’t define what engineering is in the same way that they can understand science in the school environment, so a lot more questions asked what engineering actually is and what engineers do.

Did the engineers enjoy it?

Yes. All the engineers that filled in the feedback survey would take part again and recommend it to their colleagues – see page 21. Engineers have their outlook on life changed. Lt Emma Bould from the Royal Navy was forced to confront the no-holds-barred reality that students view a warship more as an aggressive than defensive tool. “How many people have you killed?”. Matt Maddock realised that his pre-event fears of taking part were groundless and exhorted his fellow engineers to do more explaining about the world through public engagement.

Did the students benefit?

Yes. 87% of students that registered on the site actively participated by asking a question, taking part in a live chat, leaving a comment or voting. This is similar to I’m a Scientist, but still impresses us every time. Teachers found their students are now more aware of how Maths, D&T and Science studies relate to the real world. 94% of students who gave us feedback said they learnt new things about engineering – see page 27.

Did we meet our objectives?

We met two of our objectives, and the third was set too high initially as we hadn’t yet properly analysed data from I’m a Scientist as a benchmark to compare I’m an Engineer to.

Did I’m a Engineer work?

Yes. Engineers emerge from the two weeks in a state of exhilaration, energised by the questions and chats. Students are buzzing at the realisation that they can ask whatever they like and get an answer from a real person, from someone who isn’t a teacher.

What’s next?

Now it’s time to start chapter 2 and make the event sustainable and give hundreds more engineers and thousands more students the chance to get talking.

Posted on November 16, 2012 modemily in Evaluation, News | Leave a comment