Scaling - Up Process - Oriented Guided Inquiry Learning Techniques for Teaching Large Information Systems Courses

Promoting engagement during lectures becomes significantly more challenging as class sizes increase. Therefore, lecturers need to experiment with new teaching methodologies to embolden deep learning outcomes and to develop interpersonal skills amongst students. Process Oriented Guided Inquiry Learning (POGIL) is a teaching approach that uses highly structured group work during lessons so that students construct content. Each group member takes on a role for which the other group members rely on, and then the group reports back to the class. This paper describes how POGIL was adapted for use in a large first year Information Systems course. Quantitative and qualitative results suggest that students are receptive to POGIL, as the approach creates a relaxed, friendly and trusting community that is typically not present in classes that use a didactic teaching style. We provide a discussion of lessons learned with regard to using POGIL in a large lecture theatre and give recommendations for how to improve interactivity amongst the class.


Introduction
The didactic approach to lecturing where there is one-way communication from the lecturer to the students is clearly ineffective (i.e., the "sage on the stage") (Moog & Spencer (2008). Research conducted into this teaching style suggests a disconnection with the way the human brain is wired to interpret new information and learn (Bransford, 2000). Therefore, the traditional approach to lecturing needs to evolve with the advent of new technologies and teaching methodologies that are better suited to the way humans learn. The scope of the problem magnifies when class sizes become large (e.g., 200 + students).
Process Oriented Guided Inquiry Learning (POGIL) (Moog & Spencer (2008), POGIL. (2012)) is a teaching methodology that works by creating highly structured group work during lessons. POGIL "flips" the lecture so that the students are building and presenting the content rather than the lecturer. This approach develops interpersonal and communication skills amongst the students that are critical for their careers as they transition from study into the work force. POGIL has traditionally been used for teaching Chemistry to large classes to promote engagement and the fostering of soft skills (Farrell, et al (1999), Hanson & Wolfskill (2000)). Recently, POGIL was adapted for use in Information Technology (IT) classes -where there is a significant need for nurturing soft or interpersonal skills (Myers, et al (2012); Trevathan and Myers, (2013)). The Myers, et al. (2012) study involved two subjects (networking and databases) taught over a twoyear period. The study found that students' problem solving abilities, interpersonal skills and learning outcomes improved significantly. Over 85% of the students surveyed stated that they found POGIL beneficial to their studies. However, the IT class sizes in the Myers, et al. (2012) study were relatively small (i.e., up to 70 students). This paper investigates how well POGIL scales up to a class size of approximately 350 students undertaking an Information Systems (IS) subject consisting predominantly of business, management, commerce, and accounting students. We describe the course structure and strategies employed to promote engagement using POGIL. Quantitative and qualitative results suggest that POGIL creates a relaxed, friendly and trusting community that is typically not present in classes that use a didactic teaching approach. We also provide a discussion of lessons learned with regard to using POGIL in a large lecture theatre and give recommendations for how to improve interactivity amongst the class. This paper is organised as follows: Section 2 provides background on POGIL and how this teaching methodology operates in lectures and workshops. Section 3 describes the methodology for how POGIL was adapted for use in a large IS course and the way data was gathered to provide feedback on POGIL's effectiveness. Section 4 presents quantitative and qualitative results from the teaching surveys conducted, and provides a discussion on the lessons learned regarding the use of POGIL in a large lecture theatre. Section 5 gives a few concluding remarks and avenues for future work.

How POGIL Works
POGIL is a method that uses highly structured group sessions where students work on processoriented tasks to derive answers as a collective. The students work together in groups of three or four and are assigned one of the following roles: Manager, Recorder, Presenter or Reflector (optional). The use of roles enforces accountability within the groups so the outcomes are peerdriven as opposed to instructor-driven where the motivation of completing tasks is to satisfy the instructor. Each role is dependent on the other roles, so students are accountable to their peers for the function they play (D. Hanson & Wolfskill, 2000;D. M. Hanson, 2006;POGIL, 2012).
The Manager ensures that team members are fulfilling their roles, accomplishing the assigned tasks on time, and all group members are participating in the activities and understanding the concepts. The Recorder scribes the group's discussions and important aspects of the group's observations, insights and the significant concepts learnt. The Presenter delivers concise oral reports to the class. The Reflector observes group dynamics, behaviour and performance and may report to the group (or the class) about how well the group operates.
The POGIL activities focus on core concepts and encourage a deep understanding of the course material through an exploration to construct understanding while developing higher-order thinking skills. The tasks include directed, convergent or divergent questions. Directed questions can be answered directly from the information provided, convergent questions require groups to reach a consensus of the solution and divergent questions can have a range of possible correct responses. To initiate POGIL in a subject, the class must first undergo a "building the POGIL culture" stage. This is a process whereby the students come to understand the benefits of using POGIL. This is achieved though fun exercises that prove effectively to students the advantages of shared information and collaborative learning. The students also learn the rules and roles of the "highly structured" group sessions, and the expectations for each session.
Once the POGIL culture has been established, students come to class prepared for POGIL group work. While there are numerous formats for how POGIL lessons can be structured, a common approach is to have a ten-minute "mini-lecture", then a "breakaway POGIL session" for five minutes. During the POGIL session, the students will work on an exercise related directly to the lecture content. The instructor mingles amongst the groups to gauge how the students are performing. At the end of the session, two or more groups are called on to report back to the class (during a "consolidation session"). The instructor then typically recaps and/or adds to the students' answers.

Problem Motivation -"The Tumbleweed Moment"
POGIL was originally used for teaching Chemistry students in a laboratory setting over the course of a full day. The Myers, et al. (2012) study was the first time POGIL had been applied in a discipline outside of Chemistry, and in a traditional lecture theatre setting. However, the study involved smaller lecture sizes of up to 150 students. The main problem addressed in this paper is how to scale up POGIL to large classes of potentially 400 students. This paper aims to extend upon the Myers, et al. (2012) study by describing what additional teaching "infrastructure" in the form of techniques, resources, or approaches are required to make POGIL effective in a large class setting.
The biggest challenge with a large class is promoting engagement. Using POGIL does not guarantee that engagement will occur. We have dubbed the situation where engagement does not occur as the "tumbleweed moment". In American western cowboy movies, tumbleweed is a dead circular plant that is perpetually blowing around a desolate landscape. The tumbleweed is a symbol of deafening silence and hopelessness. We metaphorically apply this to the moment in either a POGIL breakaway or consolidation session, whereby the students do not respond to any prompts or engage in any manner. From the lecturer's perspective, it almost seems as if tumbleweed is blowing across the lecture theatre. This is accompanied by a feeling of failure and despair. The challenge is how to avoid the "tumbleweed moment" in POGIL. This paper seeks to provide strategies and approaches in the form of lessons learned that increase the chances of engagement.

Methodology
The trial course was an IS subject with a cohort of approximately 350 students. IS differs to IT in that, in addition to technology, it focusses on the people and procedures that employ the technology for a business-driven goal. As such, this study has a slightly different student demographic than Myers, et al. (2012) with business, commerce, accounting and management students (as compared to primarily IT students). The remainder of this section describes the how POGIL was adapted for use in the subject, and the quantitative/qualitative approach that was used to gauge how effective POGIL was throughout the course.

The Course POGIL Structure
The first two lectures in the course focused on "building the POGIL culture" Myers et al. (2012). This involved gentle interactions between students to show them the benefits of group work. Students were directed to individually draw an image based on some verbal instructions, and then rate their confidence that the image was correct (refer to the top section of Table 1). The exercise is then repeated as a group. The idea was that the group's confidence regarding whether the image was correct would be generally higher than an individual's confidence. The exercise also gets students used to the idea of working in groups.
At the start of the second lecture, the entire process was repeated with a new set of instructions for an image (refer to the bottom section of Table 1). However, students were placed under pressure by having a slightly more challenging image and less time to come up with a solution. After this exercise, the actual course content begins, and there is a content-related POGIL exercise undertaken later on in the lecture. The point of the second lecture was to reinforce the idea of group work and also to introduce students that may have missed the first lecture on what was expected for POGIL group work. Table 1 "Building the POGIL Culture" Exercises (Myers et al., 2012).
In the middle of the paper draw a parallelogram To add an element of "quirkiness" to the POGIL sessions, we employed the use of three flying screaming monkeys 1 (see Figure 1). The flying screaming monkey is a sling shot that propels the monkey through the air. An accelerometer within the monkey senses the vibration and causes the monkey to shriek. The monkey's purpose was to signal the end of a POGIL breakaway session. The monkey acted as a universal (and fun) sign that the students had to quickly focus their attention on the teaching facilitator again. The idea was that this would be a better way to regain control over the class rather than through issuing verbal commands. There was also the added "engagement bonus" that students could be hit by the monkey during the lecture. (Note that students would not be physically harmed by the monkey as it is a plush toy.) After the "building the POGIL culture" phase, POGIL exercises were introduced in every lecture.
Lectures were limited to 50 minutes. After allowances for 5 minutes at the start and finish of the lecture for "housekeeping" (e.g., announcements, admin, etc.), there was only 40 minutes remaining for actual teaching. This made lesson timing critical. Throughout the semester we experimented with two types of POGIL lecturing format: • 2 x (10 minute mini-lecture, 5 minute POGIL breakaway session, 5 minute POGIL consolidation session); or • 1 x (20 minute mini-lecture, 10 minute POGIL breakaway session, 10 minute POGIL consolidation session).
The POGIL exercises were of three types: Convergent, divergent, and directed. An example of a convergent question is a debate over competing government policies on high-speed broadband. A divergent question may be to analyse the fast food industry using Porter's five forces model. Alternately, a directed question could be "Explain how social networks facilitate viral marketing". During each exercise, a modest prize was on offer for the group that gave the best answer.

The Griffith University PRO-Teaching Project
PRO-Teaching 2 uses a developmental form of peer observation (review) of teaching that creates a collaborative and supportive atmosphere for staff to discuss and share ideas around teaching practice. Using a structured approach, data is collected from a range of sources to build evidence of teaching quality. The PRO-Teaching project consists of two rounds of peer-feedback and student evaluation (see Figure 2). Two peers are chosen (one with discipline knowledge and the other from a teaching and learning background). The peers reflect on the lecture in terms of a broad range of criteria as outlined by the PRO-Teaching project including: 1) aims and objectives; 2) content knowledge; 3) teaching techniques; 4) personal characteristics; 5) formative assessment; 6) deep learning outcomes; 7) curriculum structure; 8) learning environment; 9) teaching aids and materials; and scholarly approach. After each evaluation, the peers discuss "things that worked well", "points for consideration" and "agreed development ideas" with the lecturer. During the lectures being evaluated by the peers, students are also issued with an optional survey asking them to gauge the teacher's effectiveness from their perspective across a range of criteria.

POGIL Survey
The PRO-Teaching program is largely geared towards professional development in teaching. The surveys are designed to provide the lecturer with feedback about his/her teaching approach and aptitude. The surveys do not specifically target extended aspects related to POGIL that this study was intending to capture. Therefore, a second survey was conducted with POGIL specific questions at the end of the course. This was administered online 3 and remained open for two weeks.

PRO-Teaching Results
Table 2 presents the questions from the PRO-Teaching student survey. Students were asked to rate each question from 1 to 5, where "1 = strongly disagree" and "5 = strongly agree".

Table 2. PRO-Teaching Student Survey Questions
How effective is this teacher in:

Q1
Making clear the objectives of the lesson?

Q2
Using approaches to help you learn?

Q3
Motivating and inspiring you to learn?

Q4
Highlighting the relevance of what you were to learn?

Q5
Assessing your prior knowledge before explaining new material?

Q6
Ensuring that you received feedback which helped you to learn?

Q7
Explaining the requirements and standards of work for excellence?

Q8
Helping you to extend your knowledge understanding and skills (i.e., challenged you)?

Q9
Teaching in an organised and coherent manner?

Q10
Using feedback to improve his/her teaching?

Q11
In helping you to learn overall?  Figure 3 presents the survey results from the two student observations of the lectures. The number of respondents was 19 and 41 for observations 1 and 2 respectively. The use of paper planes was suggested by Kahn & Walsh (2006) as a fun way of encouraging feedback from students. As such, the students were asked to make paper planes out of their survey forms for the second observation. The paper plane approach doubled the response rate for the second observation.
In general, the second observation scored marginally better than the first observation. This suggests some improvement in the lecturer's personal style. The significant result is that the lectures were rated as being approximately 4/5. This indicates that the students were positively responding to the POGIL approach -particularly with regard to question 4 ("Highlighting the relevance of what you were to learn") and question 11 ("In helping you to learn overall").
Questions 3, 7 and 9 rated the most highly. These questions were "motivating and inspiring you to learn", "Explaining the requirements and standards of work for excellence", and "teaching in an organised and coherent manner". These questions are consistent with POGIL's philosophy as to Observa4on 2 what POGIL is trying to achieve. Question 6, "ensuring you received feedback which helped you to learn" rated the lowest of all questions. This may have something to do with improving the lecturer's techniques for closure on the consolidation sessions to ensure that all students do sufficiently understand the content (explained further in Section 4.3).
A shortcoming of the survey results is the low response rate in terms of the percentage of overall students in the class (5% -11%). Future work would involve further surveys across multiple offerings of the course and other measures to increase the response rate. Table 3 shows the questions for the POGIL survey. Students were asked to rate each question from 1 to 5, where "1 = strongly disagree" and "5 = strongly agree". Why do you feel this way? Could you share an example?

POGIL Survey
Q7 Overall, I feel that the POGIL group work in lectures has been a positive learning experience.
Q8 Thinking about the way your other classes at university have been taught, do you think POGIL group work could benefit other subjects? Do you have any comments about the ways that POGIL group work was an advantage or disadvantage compared to the teaching in other subjects?

Figure 4. POGIL Survey Results
There were 19 respondents for the POGIL survey. Figure 4 shows the survey results. Table 4 provides samples of student feedback when asked to comment on why they scored certain questions the way in which they did. The survey results largely support those of the PRO-Teaching survey and suggest that students were receptive to the POGIL approach. Most respondents felt that they developed a deeper understanding of the content, enjoyed peer engagement/feedback, and liked the interactive nature of the lectures. The most prominent suggestion for improvement was stricter enforcement of POGIL groups. Overall, most students thought that POGIL would be beneficial to their learning if used in other subjects. There was only one respondent that strongly disagreed with his/her experiences with regard to POGIL.

Lessons Learned for Facilitating Engagement through POGIL in Large Classes
The following are a few key points discovered through experimentation and reflection throughout the course with initiating the POGIL sessions. The points came about by casual discussions amongst the teaching staff.
The lecturer must mingle with the students. During the POGIL breakaway sessions, the lecturer absolutely must try to circle through at least part of the class. Remaining stationary at the front does not promote engagement, nor does it allow the lecturer to get a feel for whether the students are understanding the content. Mingling with the students also allows the lecturer to identify which groups have a strong answer so that they can be singled out to report to the class if the lecturer is having difficulty with getting groups to volunteer during the consolidation session. Time constrained activities may not always allow the lecturer to get very far around the class. Therefore, different parts of the class can be ventured to during different breakaway sessions in order to cover as much of the class as possible.
Ideally, the lecturer should stand in front of the lectern. The lectern can often create a mental and physical barrier between the lecturer and the students. Where possible, the lecturer needs to be in the space half way between the lectern and the front row of students. For a large lecture theatre, a wireless microphone and remote mouse (to advance through lecture slides) are required.
Strictly adhere to time constraints. Lectures were limited to 50 minutes in the trial subject. After subject housekeeping (e.g., five minutes at the start and end of the lecture), this only left 40 minutes for the lecture content. POGIL is time consuming, which limits the number of concepts covered in a lecture. We found two-three concepts to be the maximum. The time allowances need to be strictly adhered to for the breakaway and consolidation sessions.
Recap, summarise, or crystallise answers at the end of the consolidation session. After the groups have reported, the lecturer needs to bring all the answers together to close the consolidation session. The lecturer cannot just have the students respond and then quickly move on with the lecture. This leads to a situation we have termed "the void". "The void" is a moment where all the energy has been sucked out of the POGIL exercise. POGIL has not failed, but it definitely has not succeeded. The danger with not correctly closing the exercise is that some students might not have heard the other groups' answers, or the answers given are incorrect or incomplete. This creates "the void" where the students do not understand the content and the POGIL exercise has largely been meaningless.
Closure for the consolidation session can occur in various ways depending on the nature of the exercise. A class debate may not need much effort to close if the lecturer has facilitated the debate well. A convergent exercise might require the lecturer to record selected groups' answers as they are being reported (e.g., by typing the main points in a word processing document visible to students via a projector), or by having selected groups show their answers via a document camera (or other means), and then have the lecturer show the recommended answer. Bear in mind that the method of closure must be suitable for the type of exercise and must adhere to time constraints.
Visually show the POGIL exercise and specific instructions during the entire breakaway session.
The instructions can be shown via a projector and/or through the use of printed hand-outs. This allows the students to refer back to what is being asked of them to ensure that they do not deviate from the task or misinterpret what the lecturer has said. We found that verbal-only instructions for a POGIL session were less effective than verbal instructions backed up with visual instructions.
Make strategic use of prizes to encourage interaction. Awards and incentives can prompt groups to volunteer to report to the class. However, prizes must be awarded conservatively and are not always effective. Only recognise high quality answers, otherwise the value of the prize is diminished. Students will feel that they will get a prize regardless of whether they try or not. Furthermore, the prize needs to be highly publicised. That is, students need to be aware that the prize is on offer, and when it is awarded it should be done with the class's whole attention. Having the students applaud when the prize is awarded makes the recipient feel good about him/herself. In many ways peer recognition is worth more than the prize itself.
A potential situation that may develop throughout the course is that several dominant personalities may end up with the majority of the prizes. The way we avoided this was to target differing groups to report based on information gathered by the staff during the breakaway session.
Use support staff. As the lecture is large, it was found that two additional staff were beneficial for running the lesson. The following are some reflections on the use of support staff: Strategically place staff around the room. The reality is that a lecturer will have difficulty controlling all elements of the class. Support staff strategically positioned at the rear of the room can help control background chatter and reduce interference from late arriving students.
Staff must support the authority of the lecturer. They must not talk to students (or amongst each other) while the lecturer is speaking (during the mini-lecture and the consolidation session). Staff need to quickly identify groups to report and/or usher students to the front of the class when called upon by the lecturer. Supporting the lecturer's authority is critical for the cohesion and timeliness of the lecture/POGIL sessions. Unstructured or non-compliant staff makes the POGIL session adhoc and less engaging (even to the point of destroying the exercise).
Have staff actively engage the students during POGIL sessions. This is for the same reasons that the lecturer must mingle amongst the students. Support staff can be dispatched to areas where the lecturer is unable to venture to during a breakaway session.
Staff must know the exercise thoroughly prior to commencing the POGIL session. This is to ensure that the POGIL exercise is effective. It requires a staff briefing prior to the lecture.
Staff must be fully versed in the expectations of POGIL. POGIL is a high-energy activity and the students need to pick up on that energy at a conscious and subconscious level from all staff. Staff need to actively approach students, ask them how they are going, and offer guidance. Staff must not wait until a student or group asks for assistance.
Students cannot cluster at the back of the lecture theatre. There is a tendency for students to congregate in the rows that are the most distant from the lecturer. In a large lecture theatre, this can create a significant physical divide between the lecturer and the students. This situation makes it more difficult to engage students as the divide facilitates the ability for students to "hide". Lectures with lower attendance exacerbate the problem as the divide is even greater, and there are fewer potential students to provide interaction with.
Some solutions to this problem are: 1) Instruct all students to move to the front of the theatre prior to commencing the lecture; 2) Taping off the back rows so that they are not accessible to the students when they arrive; and/or 3) Have support staff act as ushers to instruct students to sit up close as they arrive -this is especially useful for late arriving students.
Force students to engage with a group. Some students try to work individually during the breakaway sessions (i.e., they are not part of a POGIL group). Students need to know that the group work is compulsory. The teaching staff can single out individuals during the breakaway session and gently ask them whether they are part of a group, and if not, suggest that they should compare notes with a nearby group (or other individuals that are also not part of a group).
Maintain consistency of lecturer. Not all of the lectures throughout the course were given by the same lecturer (i.e., there were guest lecturers from industry). There was a notable drop-off in engagement in these lectures as the guest lecturer was unfamiliar to the students. This led us to conclude that consistency of the lecturer (i.e., the same lecturer (or lecturers) who built the POGIL culture and has run all the POGIL sessions) is a key factor in maintaining engagement throughout the course.
Engagement with the flying screaming monkey. In general, students reacted positively to the use of the monkey. Many students laughed or smiled when the monkey landed near them. There were numerous occasions when the monkey was fired back at the lecturer during the lectures. This created a moment of spontaneity, which served to maintain attention.

Conclusions
This paper described how POGIL was adapted for use in a large IS course. Preliminary results suggest that students enjoyed the engagement, could understand the content better, and liked hearing what other group members' thoughts were. POGIL creates a relaxed community environment that is typically not present in classes that use a didactic teaching approach. The main contribution provided here is a series of lessons learned. We intend to use these lessons to develop an instructional framework that other teachers can use if they choose to adopt POGIL for their courses.
Future work will explore what is necessary for the process of "building the POGIL culture for staff". That is, what exercises would be required to train staff to facilitate POGIL sessions to support the lecturer. However, a limitation of the study was a low response rate to surveys, therefore, further experimentation and evaluation is required in future offerings of the course. We will endeavour to undertake studies across different courses and colleges, and will look into the effect that different genders and ethnicities may have on engagement/willingness to participate.