Re-visiting the flipped classroom in a design context

After explaining our experience with a flipped classroom model of learning, we argue that the approach brings to light the dramaturgical and mediatized aspects of learning experiences that favour a closer connection between recorded content and “live” presentation by the lecturer. We adopted the flipped classroom approach to learning and teaching in a class of over 100 postgraduate level university students, some learning at a distance, and run over two successive years. This article describes the format of the lecture recordings, class activities and assessment method. We also describe the outcome of course evaluation, and present what we learned from the process.

Such play added heat to the politics of education presuming to displace and deconstruct a set of pedagogical conventions. In fact, Derrida (according to John Caputo) advocated "Doing an 'inside job' on the institution" (Caputo, 1987, p. 234) by delivering the canonical, conventional and institutionalised knowledge base while simultaneously providing the means to subvert it (the institution and the knowledge). Such ludic intellectual tactics focussed on the lecture, though those of us in architecture and the creative arts knew about practice-led, studio-based, lab-based and a host of active learning (Felder & Brent, 2009;Freeman et al., 2014), learning-by-doing, reflection in action (Schön, 1983) and dramatic formats that departed from the conventions of the lecture (Snodgrass & Coyne, 2006).

Off campus learning
Move forward to more innocent and practical times with rapidly developing networked learning technologies and different student skills and expectations. Now deconstruction in the classroom is replaced by the flip. The flipped classroom style of pedagogy started with high school educators dissatisfied with the usual lecture format in which the pupil attends a lecture, then returns home at the end of the day and does some homework. Why not make the lecture into the homework? Pupils can "attend" the lecture as homework before the class. They can then do something else during the class timeanswer questions, have group discussions, quizzes, and activities. The flipped classroom approach simply means that the teacher supplies the course content via videos and other learning materials for the students to absorb in their own time in private at home before they come to class. Class time is then dedicated to other learning activities.
In university education, lecturers and course organisers commonly require students to prepare before coming to the classroom by reading texts or watching videos. In the flipped model, however, students watch and listen to whatever it is that would normally be delivered in an actual lecture, but in advance of the class. According to advocates of the flipped classroom approach Jonathan Bergmann and Aaron Sams, "the time when students really need me [the teacher] physically present is when they get stuck and need my individual help. They don't need me in the room there to yak at them and give them content; they can receive content on their own" (Bergmann & Sams, 2012, pp. 4-5). A blog by Kris Shaffer adds further support to the flipped classroom rationale: "put student work that requires a low cognitive load (such as information delivery or memorization) outside of class so that the time spent in the presence of peers and the teacher can be devoted to higher-order thinking and more complex tasks" (Shaffer, 2015).

Lecturing in the wild
The flipped classroom is assisted in no small part by distributed rich media technologies. It is easy now to make video recordings and upload them to a shared server (YouTube, Vimeo) to be accessed by anyone (or password protected) online. Specialised screen capture systems such as Panopto (panopto.com) mean the lecturers can record their lecture while sitting in front of a laptop, and also record Powerpoint slides at the same time. Peripatetic lecturers can go out into the field and record parts of their lecture on a smartphone attached to a tripod, a selfie stick, or propped up against a pile of rocks. There is no need to stop with a talking head and slides. Lecturers can interview people, insert clips from YouTube, edit and resort to a range of mass media production techniquesif they have the inclination, time and skills. Depending on the subject matter, students can be encouraged, or required, to do the same thing, via an assignment that involves an interview recorded on video. We used all of these techniques in our flipped classroom experiment, as well as experimenting with some specialised "vicarious learning" tutorial software (Lee, 2010). In this article, however, we wish to concentrate on learning technologies that are readily available in most Universities. Bergmann and Sams (2012) don't go this far but there is a disruptive (if not deconstructive) aspect to the flipped classroom model. In further dramatising the lecture, it brings the "mediatization" (Hepp, Hajarvard & Lundby, 2015) of professional life evident in the digital age yet again into sharp relief.
Lecturing trends towards amateur media celebrity. Recorded lectures lay the lecturer vulnerable in ways not evident in the transient, faltering and forgiving medium of the one-off stand up lecture. As well as quality content, the lecturer's errors and idiosyncrasies can be replayed and mistakes amplified and transmitted. Standing in front of a camera revives the lecturer's lingering fears about presenting in public. There are also intellectual property considerations. If material is online then it is potentially everywhere, available to anyone, and able to be copied, reconfigured, and redistributed. If students won't need to attend university at all, then will universities become mere content providers and distribution hubs?
The flipped classroom amplifies many of the challenges of online learning in general, the growth of which has led our university at least to re-evaluate what it is offering. Online learning of various flavours has led us to think more about the student experience in the round. As Kris Shaffer (2015) indicates, the flipped classroom also requires the institution to think more about the home situation of the student, or wherever else the student watches the videos, and undertakes any preparatory work. Is home life conducive to such concentrated attention? Or perhaps home-based learning percolates into cafes, pubs, foyers, libraries, parks and public transportation.

Online lecturing
The course for this trial was Media and Culture with an emphasis on digital media taught at the University of Edinburgh. We have taught the course for over 15 years. The course content requires continual updating as the technology changes and the relationships between media channels reconfigure. We emphasise design and the practices of making, using, incorporating and critiquing technologies and the cultures in which they are developed and promoted. The students enrolled in the course are from a diverse range of backgrounds but all come with an interest in design. Only a few hold design qualifications. The course is run within the Edinburgh School of Architecture and Landscape Architecture and the primary lecturer is an architect; both factors influence the expectations of the students.
When we initially started this course, digital media was a niche subject. Now the issues it raises are under discussion every day on television and radio news and magazine programmes reflecting awareness in the community at large. The effects of digital media are part of the everyday experience for most of us. Though its precursors were in evidence, ubiquitous smartphone access to social media did not exist in the 1990s. Some of the themes that were only marginal in the 1990s have gained prominence as sources of "moral panic" (Cohen, 1973), for example, Internet addiction, online grooming, isolation, digitally informed political revolutions, mass migrations, and even fears that one day machines will take over every aspect of people's lives (Kurzweil, 2005;Ray, 2011;Saville, Gisbert, Kopp, & Telesco, 2010;Turkle, 2011). The course aims to tease out new and long-standing issues while at the same time staying grounded in its original impetus in the area of design. The course supports students who are actively engaged in the professional business of using and developing digital media applications. Those of us who contribute to the course have long thought of students as co-contributors. After all, digital natives or their older siblings are starting to populate our classes (Ofcom, 2008(Ofcom, , 2016. So, we think of the student cohort itself is an excellent learning resource! Three other factors contributed to our motivation to move towards a flipped classroom model of teaching, each resulting from the globalisation of educational delivery. Firstly, we received funding to provide an online distance learning (ODL) variant of our courses, and the first year we recruited 5 students based in the UK, Ireland, the USA, the Middle East and The Philippines. Several enrolled in the Media and Culture course as ODL students. The resident student cohort also grew by nearly 50% compared to previous years, made up mostly of students whose first language is not English. A large proportion was from China. In previous years, we would record our lectures as audio podcasts and many students would report that they appreciated the opportunity to re-play the lectures to revise and to improve comprehension. So we had some support for the view that recorded lectures aid understanding. Most importantly, with an increasingly international cohort, we were keen to improve the student learning experience, to encourage deeper engagement with the issues raised in the course (O'Flaherty & Phillips, 2015), and to facilitate sharing amongst students and staff with varied cultural backgrounds.
In order to facilitate the flipped-classroom model, we pre-recorded 11 content lectures. Eight of these were recorded via HD (high definition) video on a smartphoneroughly one hour in length divided into 10-15 minute segments. They were recorded in a range of locations, some outdoors (shown in Figure 1) and with some film excerpts from YouTube inserted to illustrate something about digital cultures. The three remaining lectures were recorded using Panopto screen capture software and incorporated PowerPoint slide content. So, we explored a range of presentation styles. None of this involved professional studio recording, professional producers or editors, or specially equipped lecture capture teaching rooms. We were keen that the digital lecture delivery continued the practice of according independence to the self-reliant lecturer. This meant that we could change and adapt course content on the fly in response to changes in subject matter and responses from students. In the second year of the course we re-recorded certain segments and uploaded these to the video server. Supplementary to the video lecture content, we included links to reading lists, notes, the lecturer's blog, and short blog-style quizzes on the lecture content. All of the course material was made available online through our university's virtual learning environment (VLE). In the first year, the material was prepared throughout the semester with much of the lecture content delivered a week or so before it was needed. We adjusted content according to observations about how students deployed the material during class activities. The ODL students also took part in the class activities, albeit not in real time. We adjusted the course activities so distance students were able to participate in the process by recording videos, audio and writing short texts.
A learning technologist was also on hand mainly to assist the ODL students. That our ODL students worked independently of the live classroom gave them an opportunity to devise new ways of working together. Some of the students recruited colleagues or friends for video-based roleplaying tasks that we set. The ODL students were also asked to participate in the blog discussion that involved both on campus and ODL students.

Student assessment
As this was a course in digital media, we thought it appropriate to make digital output the focus of assessed student work. In previous years, we required students to submit a video essay, that is, a carefully-scripted 6-minute video of the student delivering on a digital media topic selected from a range of suggested essay topics. The video added little to the essay format, and markers found it easier and quicker to focus on just the written script. We doubted that the video dimension added much to the student learning experience. Therefore, we alighted on the idea of a video interview. Students were to choose a relevant topic and then select a fellow student or outsider and interview them about their topic (shown in Figure  2). They then wrote a reflective report on what they had discovered. In preparation for this exercise, we asked students to submit a short essay describing their topic followed by a list of carefully worded questions. The teaching staff then provided feedback on the content and the questions allowing sufficient time for the students to set about recording their interviews. A further submission involved a case study of the student's choosing. This was typically desk-based research into a high profile case of digital media development, use, or controversy. The lecture titles covered week by week included: post digital cultures, play and creativity, emotion and the body, voice and text, metaphor, music and sound, digital society, artificial intelligence, virtual reality and techno-utopias, ethics, audience engagement and entertainment.

Class time
The flipped classroom raises the major challenge of how to make the best use of the two hours of timetabled class time per week after students have previously absorbed the lecture content as a video before coming to the class and undertaken preparation. The class sessions were used to reinforce the lecture content, facilitate discussion on the lecture themes, encourage participation and presentation, encourage academic reading and writing, develop critical reading and comprehension skills, and prepare for the assessed submissions (a video interview and a case study). We also planned that the whole class, including the lecturers, would learn more about community attitudes to digital media and social media and their impacts. Engagement is key, and we aimed for variation in the class tasks and activities (Pohl, 2000). We felt that it helped if the students were suitably prepared for each class and if the two-hour class session was carefully structured. We booked a large teaching space with tables organised for discussion groups of 8-12 (shown in Figure 3). Each table had a computer monitor connected to a PC and a main console for sharing visual material. The space could also be divided into three smaller spaces by movable partitions. The arrangement facilitated small group discussion and the aggregation of the smaller groups into three larger groups for plenary discussions. In the first session, we divided the class into groups of 12 and gave each group a name according to the International Radiotelephony Spelling Alphabet, viz., Alpha, Bravo, Charlie and so on. This fitted the initial post-digital theme which also encouraged us to commission each group to move outside the classroom to gather video evidence on how people live with and without digital technology and to upload these short videos to YouTube. They were then to discuss these in their groups: Is there really anything in the world untouched by digital technology? Does it matter? What are the implications of your view? Then followed a session in which individuals volunteered to address the whole class with their findings. This exercise highlighted an important task for both students and course organisers that followed each weekly class. This involved collating all the work produced by the students and uploading or linking to the course pages of the VLE. With 5 students undertaking the course by distance, we were keen to build up this class resource and to maximise the opportunities for interaction between the resident and the distant cohort. At least, we believed, they would be aware of each other's work.
The following week involved another structured programme of creative activity, this time focussing on writing and storytelling as a group activity. We will not say more about these initial sessions as the objective was to move the class towards enhanced skills in reading, writing, discussing and presenting critically. Working in an art and design context, we feel we have little difficulty in keeping students gainfully occupied in varied creative group activities. The challenge for us was to lead into sophisticated group discussion informed by the literature and on topic.
To this end, we led the class towards a standard workshop or conference format. The groups would discuss allocated topics or readings, document their findings on paper flip charts and then present to a larger group. It is also usually the case that participants fall into the same roles every week, with some having more to say that others. Half way through the semester we had a session where students were given name tags to wear indicating a new role for each: a chair, note taker, two presenters, an experienced coach who would offer advice and encouragement before and during presentations, and others primed to ask questions after the presentations and keep the discussion moving. After the first such session we found that in subsequent weeks students would habitually share roles within their groups and seemed more confident as a result. We should add that presentations were recorded on smartphones and the video files uploaded to the VLE server as a record, a resource, to encourage familiarity with the video medium, and for sharing with the ODL students.
Each week's class would also include a novel engagement activity. For example, in discussing crowd-sourced creativity, we required the class to estimate the number of dried beans in a jar to test the theory that the average of the guesses is close to the actual number. (It wasn't.) The week in which we talked about artificial intelligence, we asked the class to discuss questions they could put to "the oracle," to write these on pieces of paper and put them in a question box. Questions were taken out at random and put to the lecturer (the oracle). After he answered, some volunteers would put the question to various online chat bots with amusing results. This and other approaches were on topic especially in light of the lecture's focus on the Turing test.
As a further example, during the session on techno-utopias, students were primed to review a segment of the film Metropolis, to act it out and record it (shown in Figure 4). The teachers then assembled the sequence to reconstruct the film. This proved not only an amusing and surreal exercise in absurdist collage but provided a way of gaining familiarity with the issues raised by an iconic film as well as instilling greater confidence with the video medium. In the following year, we included a live demonstration of the use of head-mounted EEG (electroencephalography) techniques for recording supposed emotional responses to visual material (shown in Figure 5). A student volunteer watched an excerpt from the film Lord of the Rings: The Fellowship of the Ring (2001), while the rest of the class observed fluctuations in emotional parameters on screen, and then conducted their own analysis of the film segment.
As preparation for the assignment, we also had a session in which the lecturer set up a mise-enscene and interviewed and recorded a student volunteer to demonstrate the production of a simple and effective interview. The course coordinator was aided by two other staff members who would facilitate the larger group discussion sessions.

Evaluation
There were four main sources of data for assessing the strengths and weaknesses of the course and our application of the flipped classroom model: (1) reflections by the teaching staff or the lecturers' perspective; (2) attendance records taken in class using attendance software; (3) results of assessed student work; and (4) formal feedback acquired online via the University course monitoring and feedback system.

The lecturers' perspective
The flipped classroom format enabled the teaching staff to cope with a large student group. Recording and editing of the lecture content could take anywhere between a couple of hours to a whole day depending on the approach taken. We expect the lectures to have a shelf life of a couple of years before they need refreshing. Where lectures were broken into 10-15 minute segments, however, we expected that piecemeal updating would be possible during the subsequent iteration of the course. Initial ad hoc comments from students about the lectures were very encouraging which bolstered the lecturer's enthusiasm for creating further content. Some of the overseas students seemed to enjoy the travel dimension of the lectures, as several lecturers were filmed in interesting places in British towns, museums, the countryside or on other campuses.
Throughout the semester, the teaching staff noted how much more engaged the students seemed to be than in previous years. Attendance was high and there was certainly a lot of group discussion. The interviews and reflective reports were intriguing to watch and read, and indicated a level of sophistication we had not observed before. We had a team of markers to assess the student work. Not all markers attended the class but one remarked "the work seems much more connected with something the students want to write about and explore and it's generally easier to read and get into." He followed this with a comment that not all themes from the lectures were referenced in the student work. It is perhaps easier to interview someone about Internet addiction, social isolation, emotional machines, if machines will take over the world, and other high profile mass media themes than how digital design is affected by metaphors, Marshall McLuhan's theories about cultural transformation, and the play element in creativity. So, perhaps the interview format skewed the student's interest towards mainstream concerns.
We occupied a modern style teaching space, but of an earlier generation. It was slightly cramped for the numbers in the class and in the first year of this experiment the building was undergoing renovation. So the windows were blacked out. I mention this as one of the causes of concern in the feedback from students focussed on the condition of the room and the lack of space. The environment is clearly important. In previous years we had tried in vain to encourage group discussion in a lecture room with fixed seating on a raked floor. A large open space with tables and chairs that support group discussion is clearly preferable.

Attendance
It is possible to monitor page views via the VLE and video views via the Vimeo or the University's video server statistics pages. In the first year of the experiment, the numbers of video views tapered to about half towards the end of the 11-week course as did the numbers attending the class. That is not unusual in any course but disappointing as we hoped for 100% engagement in the lecture content at least. The measures of video views were reviewed several weeks after the course ended and the assignment was submitted. So, there was no evidence that students who missed out on their weekly lecture caught up later. Attendance for ODL students was monitored via their engagement with the weekly activities. In the case of the ODL students, they could not have successfully completed the activity without seeing the lecture recording or the recording of the class discussions.

Student performance
We had the marks for the assessed submissions, a class quiz conducted in the last session, and the attendance record. Using simple correlation analysis on an Excel spreadsheet, there appeared to be no correlation between attendance, page views, the quiz and final performance. A few students with poor attendance performed very well and some students who attended fully performed poorly. Hopefully, we cannot infer therefore that the lectures and classes were unnecessary. It is worth stating the learning outcomes (LOs) presented in the course documentation here. On completion of this course, the student would be expected to: • LO1 -Contextualise developments in digital media, technologies and devices in terms of critical theoretical and philosophical debate; • LO2 -Formulate and communicate coherent opinion and insight on key aspects of digital media and culture; • LO3 -Apply digital social media in its various forms to present and respond to discussion and debate about digital media; • LO4 -Write to a suitable standard, deploying the conventions of academic scholarship, including the appropriate use of evidence, sources and references.
We subscribe to the view that students have different learning styles and that, for some students, the LOs are adequately reached by viewing videos or just reading. For others, more is achieved by discussion and interaction with classmates. It is also possible that the assessment regime was not adequately tuned to the lecture and class content -an observation that appeared in the student feedback. Perhaps the lecture course content and assessment needed to form a better match with the learning outcomes.

Feedback from students
Feedback from students features prominently in any course assessment. The response rate to the centrally administered course evaluation questionnaire was ~70%. The approval ratings for the various aspects of the course were at the high end of the scale, and overall at around the 4 out of 5 mark. The evaluator "Overall I am satisfied with the quality of the course" averaged 4 with a wide spread while "I would recommend this course to other students" averaged a little lower and with an even wider spread. We take the spread (though skewed towards the positive side of the scale) for both these responses as indicating divided opinion on the merits of the flipped classroom approach and our implementation of it, some of which may be due to unfamiliarity with active learning approaches amongst the cohort.
The comments field of the student questionnaire provided the most insightful and critical feedback on the course. About half the respondents provided comments. Amongst the positive comments we read: • The section of the questionnaire labelled "Areas for improvement" yielded some critical comments about the approach: • I found it odd that we had to watch/ do about 1/2 hours worth of work before coming to the lectures but then didn't really do much when we got there apart from some activities that were sometimes useful but also sometimes not. • The weekly "discussion" sessions were just not working for me. Go back to traditional lecture (but in a small group, 80 students is not MSc size) or seminar where a particular paper is discussed in detail.
These comments accord with some of the negative responses reported by O'Flaherty and Phillips (2015) though, in our case, the criticism was directed at the lack of a perceived link between the lecture content and the class activities.
One set of comments pertained not just to the flipped model, but its implementation: • The "inverted classroom" type set-up was not used effectively at all. In-class activities often bore little relation to video lecture content This respondent also highlighted language issues in the class: • Different groups end up discussing different things, and the presentations at the end of each discussion do not serve to inform groups of what the other groups discussed, partially due to some groups having few to no fluent English speakers, and the policy of making sure everyone must present. Someone felt that the class period was just a time-filler: • It felt like the course was just there to keep us busy.
It is interesting that none of the negative comments related to the recorded lecture content. Rather, they highlighted an uncertain relationship between the lectures and the class activities, the main outcome from this learning experience.

Lessons learned
As with any course, there are lessons to be learned relating to course content, the size of the class, and the space in which the teaching takes place. But here we will focus on the "flipped" aspect of this particular course. Early in the course, we heard informal feedback that there was too much work to do in the course, which included watching videos, reading notes, writing blog comments and reading papers before the class. This is a common student misgiving about the flipped classroom model also reported by O'Flaherty and Phillips (2015). What we took to be a rich repository of resources for students to dip into was seen by many students as overwhelming. In an insightful lecture on technology-enhanced learning using the flipped classroom approach, Abelardo Pardo Sanchez from the University of Sydney Faculty of Engineering and Information Technologies emphasised that flipped classroom course leaders need to make clear to students what is expected of them as preparation for the class, perhaps even with step-by-step instructions (Estévez-Ayres, et al, 2015). Otherwise we are confronting students with too many choices including deciding whether they really need to go to the class as all the relevant material is online. From their scoping review of the flipped classroom approach, O'Flaherty and Phillips (2015) concluded that "instructors need to redesign their curriculum so that the pre-class activities are integrated better into their F2F classes with active learning pedagogies so students understand the model and are motivated to prepare for class" (p. 93). We feel that such clarity is needed in the organisation of the course material. Bergmann and Sams (2012) advised that having dispensed with the need to deliver a lecture in front of the class, the teacher can circulate within the classroom and answer questions from struggling students. This approach is perhaps applicable in the case of their domain of Mathematics and Chemistry, but our discursive subject matter invited other approaches.
The value of the video content was not in question but, as we predicted, the class activity provided the greatest challenge. In this case, we were uncompromising in delivering the flipped classroom model. The two-hour class sessions contained no lecture material at all. This was not a hybrid implementation. In any case, the particular space did not really invite attention on the lecturer as the students were arranged in groups sitting at tables. But we have lectured under similar conditions before, and the spatial arrangement can work adequately, especially if the lecturer uses a roving microphone. Drawing again on the dramaturgical metaphor, the lecturer in that setting becomes a cabaret performer, an after dinner speaker, or a roving magician.
We can draw lessons from the performing arts. Observers have examined the critical relationship between recorded and live performance in the music business (Holt, 2013). Audiences attend live concerts to see and hear entertainers perform the same material they could experience through YouTube videos. Prominent academic speakers also attract live audiences even though people can read their books or watch them on TED videos and even if the live lecture delivers similar content. By several accounts, distributed recordings increase the appetite for live concerts and public talks which, in turn, becomes the main revenue stream for many musicians and celebrity speakers. Translating this emergent relationship between the recorded and the live to the more routine world of the university lecture, it can be contended that live lectures delivers something that videos cannot. Apart from content, going live at its best amplifies the potential for: emotional engagement within an audience, empathy between speaker and audience (Krznaric, 2014), and setting the mood such that audiences are receptive to content. As we have explored elsewhere (Coyne, 2016), gathered audiences cultivate and share moods and contribute to a receptive "frame of mind." Distributed audiences and co-learners operating online can achieve this, but bringing people together provides immediate access to moods conducive to learning.
In light of that proposition, to repeat or extend the video lecture content may not be appropriate or efficient, but an improvised twenty-minute summary of the lecture content or its highlights might enhance the learning experience particularly if it includes responses to questions from students confident to speak out within a large class. In the first year of our implementation of the flipped classroom, we feel we missed out on this opportunity to develop rapport and enhance learning by capitalising on the lecturer's presence in the class and associating the lecturer's presence more strongly with the lecture content. It seems the students did want to hear from the lecturer "live" rather than see the lecturer as an organiser of activities.
In the second year of the flipped classroom experiment we adjusted the class format accordingly. Each session began with a 20-minute cabaret style delivery by the lecturer, who summarized the lecture content and conducting a dialogue with those students prepared to speak out to the whole class. This worked very well and students reported favorably on this innovation. We also introduced other revisions into the course including a closer coupling between work submitted for assessment and the course content.
There were lessons too from our experience with ODL students. We discovered that ODL students prefer to watch recordings of lectures where they can see the lecturer in the video as well as the slides, screen interactions, or other material. This became evident through both the formal and informal feedback we received from the ODL students. In this occasion, the Panopto lecture delivery system proved to be very useful but this option worked only when lectures were delivered live. Subsequent experimentation in other courses involved a mix of live and pre-recorded lectures. When pre-recorded lectures were introduced later in the semester, the ODL students were already familiar with the main aspects of the course, with the voice of the lecturer and way of presenting. A healthy approach for both on campus and ODL students would be a combination of live lectures, pre-recorded lectures and flipped classroom sessions. The latter would happen in the second half of the semester when the students are more familiar with the main aspects of the course as well as with each other. This way they work better in a group. We have also taught courses involving ODL students who happen to be on campus. This proved to be the case for on campus and ODL students who engaged in group activities.