ON USING POPULAR CULTURE TO ENHANCE LEARNING FOR ENGINEERING UNDERGRADUATES

This paper reports on an investigation into the effectiveness of popular culture for engaging university students in the learning process. A framework was set up for the investigation, with this framework involving two competitions, namely “Design Idol” and “The Biggest Loser”. Design Idol was embedded into a 3rd year electronic circuit design unit at QUT, while The Biggest Loser was incorporated into a 1st year electrical circuits unit at the same university. Assessment results showed that after these competitions were introduced there were improved learning outcomes.


Introduction
It is well known that learning is enhanced when students are motivated, when they are actively engaged, and when they have strong interaction with peers (Hake, 1998), (Felder, 2004). One of the key challenges in education, then, is to create learning strategies which motivate students, and which facilitate engagement and peer interaction. As part of the quest to find motivational tools to expedite these outcomes, researchers have looked to the natural interests of students. There is little doubt that computer games fall into the natural interest area of many young people. Accordingly, researchers in various streams of education have explored the use of games and role play in learning (Kafai, 1998), (Rilstone, 1994), (Harvey, 1985). The latter tools have met with some notable successes. In (Charlton, 2005), for example, there was an investigation into the use of games for helping to acquire reading skills. Eight elementary school children with learning disabilities were tested in learning scenarios with and without educational games. The games were found to consistently accelerate the acquisition of reading skills for all of the children. The use of educational games does not automatically guarantee heightened learning outcomes, however. Investigations conducted at MIT found that many educational gaming initiatives had failed, due partly to the poor quality of these games when compared with commercial entertainment counterparts (Squire, 2002). This paper considers an alternative to computer games for taking advantage of students' natural interests. Specifically, popular culture is used to inspire learning design.
Given the heavy exposure of most university students to popular culture (via television and other such media), it is of interest to know if this culture can be used in a positive way to enhance student motivation, engagement and learning. A study was done in 2004 to investigate whether or not the introduction of popular culture (as well as media and other new technologies) into the curricula of young children could improve student learning (Marsh, 2005). The children were 6 years of age or less. The findings of the study were positive -learning did improve. The specific question addressed in this paper is whether a similar effect exists for university students, and in particular for engineering undergraduate students. Accordingly, this paper investigates the hypothesis that learning designs inspired by structures within popular television shows can have a positive influence on the motivation, engagement and interaction of engineering students, and can thus contribute to improvements in learning.
To test the hypothesis two new learning designs were developed which incorporated structures adapted from popular television shows. A "Design Idol" competition was used for the third year engineering Introduction to Design unit at Queensland University of Technology (QUT), while a "Biggest Loser" competition was used for EEB213, a first year Electrical Circuits unit at QUT. These new learning designs were then trialled on students, and evaluations were conducted. Both quantitative and qualitative data was collected and analysed. The findings were positive -the competitions were seen to enhance student performance.
The new learning designs are outlined in the following section. In the subsequent section quantitative and qualitative results are presented, followed by analysis and discussion. In the final section, some conclusions are drawn.

The New Learning Designs
Third year Engineering Design unit In order to obtain an objective measure of the effectiveness of the new initiative in this unit, a control learning design was set up and used for the control cohort. Student performance in this group was then compared to the performance of students who were exposed to the new competition based learning design. A description of both control and competition based learning designs is given below.
Control learning design Students were required to work in teams of three and to design and build three electronic products -a digital motor controller, a digital voice recorder and an electronic siren. These projects were weighted at 20%, 40% and 40% respectively of the overall marks for the unit. All student teams were required to compile oral and written reports on their projects. The oral assessment was done on an individual basis, while the written assessment was done at a team level. 20% of the marks for the unit were allocated to the oral reporting and oral examination, with the remainder (80%) being allocated to the written report. There was no written examination.
The unit resources included project specifications, notes, subject guide (with assessment outline), help videos, an interactive discussion forum and useful weblinks. These were all available on QUT's on-line learning system. Regular weekly help sessions were provided, so that students could get assistance in circuit troubleshooting and other circuit design/implementation problems.
Competition based learning design The "Design Idol" competition based learning design built on the control learning design. It used the same projects, and still required students to work in groups of three. There was still 20% of the assessment dedicated to oral assessment and 80% to written assessment. The projects were still weighted as 20%, 40% and 40% respectively of the overall marks. The unit resources and help session arrangements were similar. Again, there was no examination. A new dimension was added, however, by embedding the assessment into a competition based on the enormously popular "Pop Idol" television show. Pop Idol first appeared in Britain, but has since been franchised to many different countries around the world, including Australia. The Australia variant of the show is called "Australian Idol". In this television show, a large number of contestants enter a singing competition, and these contestants are slowly "whittled down" until there is just one left -the Australian Idol. The process of selecting the winner engages a wide variety of people over a substantial period of time. The level of engagement is reflected in the television ratings, which are generally very high. There is engagement of the contestants, judges, sponsors, and most importantly a very large viewing audience. There is a strong mentoring element to the show, with judges providing frequent feedback and advice. Much of the appeal of the show lies in the fact that many people, not just the ultimate winner, benefit in some way from the competition.. Hopeful singers get a chance to audition, to get feedback, and to receive national TV exposure. Viewers benefit by having the opportunity to hear talented singers perform regularly, and by sharing in the "journey" of the contestants. Viewers also have the opportunity to influence results, with the winner being selected by a viewer poll. The ultimate winner and various other contestants benefit by obtaining financially lucrative recording contracts, The sponsors benefit though the very substantial exposure available through the show, The Design Idol (DI) competition at QUT sought to draw on the best aspects of Australian Idol. DI had an ongoing mentoring process via regular help sessions where students received help and encouragement in their idol journey. It had a finale to showcase the best projects. This finale was designed to engage not only the finalists, but the rest of the class as well. This engagement of the wider class was achieved by having students select the winners (by voting) and by having students prepare placards for the finale. The finale was conducted in a very playful way, with abundant jokes and many spontaneous interjections by staff and students. This playful aspect to the finale was in line with recent research showing that there is a strong link between playfulness and cognitive development (Iwaniuk, 2001). The details of the DI competition and its context within the third year design unit are provided in the following paragraphs.
When the students had completed the first two of their three projects, all student teams were required to attend DI "auditions". The auditions incorporated the same oral assessment process used in the control learning design, but they also served as a mechanism for selecting the DI finalists. The four best performing teams gained entry into the DI finale. Students who were selected as finalists were given a 5% bonus, while the ultimate winners were given an 8% bonus. The winners were selected at the end of the finale by anonymous peer voting. Oral formative feedback was also given during the finale by a panel of three (academic) judges.
The competitive nature of DI was used to simulate the competitive real world nature of the engineering industry. Students were asked to embrace this aspect of design by being asked to build into their products something which would give them a competitive edge. There were many ways that this edge could be realised in the projects, and so the projects were more ill-structured than they were in the control learning design. This aspect of the learning design was in line with a Project Based Learning philosophy and with a constructivist approach, where students are given scope to construct meaningful learning for themselves (Savery, 1995).
To account for the open ended aspect of the learning design described in the preceding paragraph 10% of the marks for the written report were specifically set aside for competitive edge. The finalists were additionally required to create i) a video demonstration of their working projects, and ii) a retrospective video of their idol journey, both of which were shown during the finale. The finalists were asked to compile videos which would meet the criteria of i) convincing the DI finale audience (i.e. their peers) of the technical quality of their work, and ii) engaging the audience. No criteria for compiling the videos were given to the students apart from these and so the finalists had a fairly wide scope to express themselves. Students were deliberately given a small amount of time (a few days) to compile the videos so that they would not spend too much time on them and neglect pending assignments in other units. It was envisaged that the compilation of these videos would give students an opportunity to demonstrate a number of important generic capabilities such as team-work, time management, multi-media communication skills and creativity. The video compilation tasks were also an attempt to give students practice in marketing their products, as might be required in a real world scenario. Some of the videos created did indeed demonstrate these skills, especially considering that they were created in just a few days. The winning team, for example, created a retrospective video which was a "send-up" of the Diary Room from the Big Brother television program. On this video the various group members would regularly come into a room, sit in a chair, voice complaints to the video camera about their team-mates, and then leave. The complaints made during these Diary Room sessions were typically based on real issues that had occurred in the group, but exaggerated to hilarious effect.
Regrettably, it was only the finalists (about 13% of the class) who had the opportunity to create videos, largely because of resource limitations -only one video camera was available to prepare the videos. Nonetheless, the other students were encouraged to creatively express themselves in other ways. Some, for example, made placards that they waved around during the finale. Others made spontaneous comments from the audience. The somewhat unconventional DI setting also enabled all students to be a little freer in their creative expression than they might have been in a traditional classroom. The finale was hosted by the unit co-ordinator, but in an alter-ego as his very colourful twin brother, Kevin. Considerable effort went into the script and props for the competition. Figure 1 presents a diagrammatic view of the Design Idol competition. A learning design which uses some similar design components to the Design Idol initiative can be found at (Learning, 2006).

First year Electrical Circuits unit
Control learning design The first year Electrical Circuits unit aims to help students acquire the fundamentals in electrical circuit theory. In the standard learning design for this unit, there were four assessment items. These were labs (5%), assignment (20%), mid-semester exam (25%) and final exam (50%). All were assessed on an individual basis. The unit resources included a textbook, lecture notes, subject guide (with assessment outline), a multitude of help videos (full of worked examples) and useful web-links. These were all available on QUT's on-line learning system. Regular weekly help sessions were also provided in addition to lectures, so that students could get help with problem solving.
The competition based learning design: The competition based learning design in this unit differed from the standard learning design in one key way. A Biggest Loser competition was incorporated into the unit as a supplement to all the existing components of the standard design. This competition was based on the popular reality TV show of the same name, in which a group of over-weight contestants seek to out-perform each other in losing weight. The contestants have a personal trainer to help them with weight loss strategies and must attend "weigh-ins" at the beginning and end of the competition. The biggest loser is deemed to be the contestant who has lost the most weight during the competition.
In the adaptation of the reality TV show to the Electrical Circuits unit, the biggest losers were declared to be those students who had lost the most mis-understanding between the mid-semester and final exams. i.e. the biggest losers were the students who improved the most between the mid-semester and final exams. For this competition four 6% bonuses were made available. To be eligible for this bonus all aspiring "losers" were required to enlist a personal trainer and to attend a preliminary and final weigh in. The personal trainer had to be a fellow student in this unit, and the preliminary and final weigh-ins were the mid-semester and final exams respectively. The four 6% bonuses were shared equally between the losers and their personal trainers. i.e. each loser earned a 3% bonus and each successful trainer also earned a 3% bonus. Students who wished to be eligible for one of the bonuses had to register on the interactive discussion forum. The team (of aspiring loser and personal trainer) had to be declared, although the roles of each of the students in the pair did not have to be specified. The BL competition process is depicted diagrammatically in Figure 2.
The competition was designed to facilitate students helping one another. This aspect of the design was motivated by research which shows that learning tends to improve when effective collaboration occurs and a sense of community is developed (Vygotsky, 1978), (Hake, 1998). The competition was thus in accord with sociocontructivist philosophy. There were elements of behaviourist theory motivating the competition as well. Rewards (or bonuses) were provided to elicit the desired (collaborative) behaviour.
The class was briefed on the nature of the competition about two and a half weeks before the final exam and they were given some rationale as to why the competition was being used. The class was then asked whether or not they wanted the competition to be used in the unit. The overwhelming majority of the class wanted it used, and so it was. 42% of the enrolled students registered for the competition. This article will henceforth refer to those who did register as the BL cohort. It will refer to those who did not register as the control cohort.
Because of the way the competition was designed, a number of the highest achieving students realised that their best chance of securing a bonus was to work with students who had not done well on the mid-semester exam. Thus there was a tendency for the highest achieving students to want to help those students who had not previously performed well.

Results and discussion
Third Year Engineering Design Unit: As alluded to earlier, the control and competition based learning designs were trialled on two separate student cohorts in EEB584, with a view to comparing outcomes. The control learning design was used for students in the summer semester of 2004/2005, while the competition based learning design was used for students in the first semester of 2005. The assessment (both oral and written) was performed by the same independent tutor, who was not aware that a comparison was going to be made. The comparison presented in this section was made between the overall results for the first two projects in the unit. i.e. the projects completed before the DI finale took place. The third project was excluded from the DI competition.
To make inter-semester comparison meaningful there was an adjustment for possible differences in the ability levels of the cohorts in the different semesters. This adjustment was done by first determining the quantity, D, for each student, where D was the difference between the student's score in this unit and their mean percentage for all completed units in their course. The class mean for D in the summer semester (non-DI mode) was found to be 10.29, and in the first semester (DI mode) it was found to be 12.06. (Note that students score much better on average in this engineering unit than in other units because it is fully project based and students devote a lot of time to it). A T Test was performed to test the significance of the difference in the means. The T Test that was performed provided a significance level (p-value) of 0.06. The DI competition was deemed to have contributed to a significant improvement in student performance.
In addition to the quantitative results, there were a number of qualitative indications that the competition was effective in motivating and engaging students, and in facilitating peer interaction. These indications are described below.

1.
Student representatives in two different independently organised Staff-Student Liaison Committee meetings reported positively on the DI competition (QUT, 2005). The minutes of one of these meetings (on May 5th, 2005) stated that: "Electrical Design Idol Competition -this competition took place last Wednesday and students were impressed with the coherence of the group. It was noted that [Author Name] had implemented teaching strategies that encouraged innovation, competition and incorporated assessment. The group dynamics was very impressive and the whole class worked well together". 2.
One of the staff who served as judge in the DI finale commented that students were being engaged by the competition and was keen to get similar engagement in their unit. That staff member then embarked on a similar popular culture initiative and a "Survivor, QUT" competition resulted.

3.
There was much positive oral and written feedback from students on the competition. Some sample written comments were: "Great learning atmosphere", "Awesome work. Made the subject good fun", "I really enjoyed it", "Keep Kevin as the host", "Love the earing". 4.
There was some positive feedback from Industry, who reported that they thought the competition was a good idea for motivating students. This feedback occurred even though there had been no explicit attempt to promote the competition in industry.
Some reflections from the unit co-ordinator, who had taught in Introduction to Design for many years, also seem pertinent at this point. He noticed a number of changes in students under the DI learning model. Firstly, the standard of project work appeared to be significantly higher within the DI learning mode. Secondly, DI seemed to help the attitude of students toward their work. A proliferation of positive comments (some of them included above) indicated that students appreciated the fresh new approach in DI and they appreciated the effort that had gone into creating it.
First Year Electrical Circuits unit: 42% of the students in this unit registered voluntarily to be part of the BL competition and the other 58% did not register. Both the mid-semester and final exams (which were used to determine the BL winners) were marked independently by staff who had not been involved in the BL competition design or development. Once all marking for the unit had been completed, the final exam results were compared with the midsemester exam results, and a statistical analysis was performed. The results of the analysis are presented in the ensuing paragraph.
The average mid-semester exam mark for the BL cohort was 68.77%, while for the control group it was 67.54%. This relatively small difference between the two group means was not statistically significant. (Note that there were relatively large standard deviations in both the registrant and non-registrant groups: 11.44 and 13.11 respectively). The significance level (i.e. the probability that there was no authentic difference in means) was 0.89. Measured across the entire class, the average percentage score obtained on the final exam was found to be lower than on the midsemester exam. This was probably due to the fact that the final exam tested more difficult subject matter. The average decrease in scores from the mid-semester exam to the final exam for the BL cohort was 0.66%, while for control cohort it was 16.4%. The difference in outcomes for the two groups corresponded to a significance level of 0.004. There was thus a remarkable difference in performance between the two groups, with the BL cohort doing much better than the control cohort. The BL cohort were also seen to be much better represented among students who made "major" improvements (with a "major" improvement being deemed to be an improvement of 9% or greater). About 14% of the class achieved major improvements, with 89% of these being in the BL cohort and 11% being in the control cohort. That is, based on the study reported in this paper, the BL cohort members were approximately 8 times as likely control cohort members to achieve major improvements.
The results presented in the previous paragraph show a strong improvement in performance for the BL cohort compared to the control cohort. One has to concede, however, that this might not be the only factor contributing to the difference in outcomes. An alternative contributing factor could have been that the two different cohorts were formed by self-selection. It is possible that the offer of a possible bonus had attracted the more motivated students to the BL cohort. If the BL cohort was more motivated, though, this did not manifest in a statistically significant difference in the mid-semester exam results, only in the final exam. i.e. the difference only manifested statistically after the BL competition was introduced.
Some qualitative observations about the competition are also useful. The discussion forum (which was used to register for the BL competition) became a forum where students started to volunteer to help one another, even when there was no potential bonus involved. For example, a few of the high achieving students posted notices when they were going to embark on a study session and issued an open invitation to any other students to join them. The concept of helping one another appeared to capture the students' imagination. Some oral and written student feedback was also obtained, the vast majority of it positive. Some of the comments from students were: 1. "That buddy system worked well". 2.
"when I had to explain something to my friend I had to explain it in different ways until he could understand it with his method of learning. During that process I gained more insight into what I was explaining and that helped me understand it further" 4.
"having to re-explain something or resolve a question definitely helped me refresh my memories and gave me the chance to evaluate on sections that I find weak".
The above student comments suggested that for some students at least, the peer interaction process was helping the learning.
From a staff perspective, BL was pleasing in many ways. The competition was i) easy to implement, ii) fun to run, iii) useful for improving student performance, and iv) an effective stimulus for developing collaborations with other staff (because of the interest the competition generated).

Discussion
In both units where the competitions were trialled, quantitative and qualitative results indicated improvements in learning. These early investigations have supported the hypothesis that structures inspired by popular television programs can be used to improve student learning. Further studies on different cohorts need to be performed in order to determine how repeatable the results are.
One of the surprising outcomes from this investigation was the finding that the competitions elicited the engagement of fellow teaching staff members as well as students. Some of the indicators of this engagement were that:

1.
Some staff within the Engineering Systems School became so enthusiastic about the DI concept that they approached the unit co-ordinator to be given some role in the competition 2.
Motivated by the DI experience, a fellow academic undertook a similar initiative (namely, "Survivor, QUT") in one of their units in the second semester of 2005.

3.
Several academics communicated their intention to adopt either the DI or BL concepts in their own unit(s).

4.
A number of invitations were issued for staff involved in the competitions to give talks about their competition experiences.
It is important to consider the resource implications for running the competitions described in this paper. Substantial effort was required for planning, implementation and follow-up of DI, but very little effort was required for BL. The substantial effort for DI, however, was made palatable by the fun factor. That is, the competition was fun for the staff to organise. The burden of effort also decreased as more staff became involved.

Conclusions
This paper has investigated the effectiveness of competitions which employ the popular culture factor for enhancing student learning. The finding of the investigation has been that student learning did improve when competitions based on structures inspired by popular culture were introduced. An additional (and somewhat unexpected) finding was that these competitions also increased inter-staff engagement and collaboration.