ASSESSING GROUP INTERACTIONS ONLINE : STUDENTS ’ PERSPECTIVES

The purpose of the following article is to discuss the integration of computer mediated communication into a French writing course and to report on the assessment methodology used in order to gather students’ perspectives. The online course component was introduced in the Fall of 2003 in order to enhance students’ learning by introducing collaborative writing projects, and as a result, to encourage them to write on a regular basis. The intent of the new pedagogical paradigm was to create a student-centred environment where socio-constructivist and cognitive principles would be infused into the teaching of French writing (L2 setting). A design experiment methodology based on students’ perspectives was used to continuously test and refine the online component of the course. This process assessment helped in the development of our final course model and tested the quality of our teaching.


Introduction
Web-based learning and effective teaching in higher education may not automatically go hand in hand.Too often, the technological imperative felt by some and challenged by others serves as a basis to discredit the opportunity that technology can bring to the learning environment.Many faculty members have seen their initial rush of enthusiasm followed by dark periods of technical frustration (Van Gyn, 2004), and so they become wary of IT miracles.However, as noted by Bates and Poole (2003), 'such arguments should not blind us to the genuine benefits technology can bring when used wisely ' (p. 11).Indeed, in the more particular case of language learning, online practices can bring new experiences to the learner in terms of linguistic interaction, cognitive patterns, as well as social, cultural and intercultural competences.
Over the last decade the development of computer networks, computer mediated communication (CMC) as well as course management systems such as WebCT, Backboard, Firstclass or Desire2Learn have shaped new pedagogical concepts and helped instructors to develop new, networked language practices.Today the role played by computers and CMC practices allows students to become more active and engaged in their learning.This article summarises a project introduced in an advanced French writing course where an online component was added to the traditional curriculum.Here we focus on the following aspects: (1) the research motivating our choice of technology for learning to write in a second language; (2) the methodology used to assess the impact of this pedagogical tool on students' learning; and (3) our findings and reflections of the assessment process.

Literature Review
In the last few decades, research relating to the use of computers for second language learning has focused largely on computer mediated communication (CMC) and computer assisted classroom discussion (CACD).These research projects tend to concentrate on three main aspects: (1) the amount of student interaction; (2) the linguistic patterns of interactions; and (3) the impact that these asynchronous or synchronous forms of communication may have on students' learning, and more specifically, on students' writing (Warschauer, 2004).
Research findings related to CMC have shown that this method can positively affect students' learning on both cognitive and affective levels (Beauvois, 1992;Bonk & Cunningham, 1998;Warshauer, 1996).In the case of synchronous writing, participation can easily increase to 100%.One main benefit of CMC in asynchronous writing environments is that it promotes collaboration amongst students and reduces inhibition towards communication.CMC can also better promote linguistic accuracy (Kelm, 1992(Kelm, , 1996)), particularly in asynchronous writing environments where students have the opportunity and the time to edit their text (Sotillo, 2000).Drawing on interactionist theories, recent projects have focused on the negotiation of meaning in the patterns of interaction that occur in online discussions, in particular synchronous discourse (Kern, Ware, & Warshauer, 2004).
Much of this research effort and its findings reflect, in part, conclusions from other studies (Bonk & King, 1998;Bonk & Cunningham, 1998;Kelm, 1996) showing that computer supported collaborative learning (CSCL) can enhance the integration of key pedagogical concepts and reflect theories such as Vygotsky's sociocultural theory of cognitive development.The notion of learning as a social activity seems particularly suited to the CMC environment where the computer allows learners to connect to each other and work collaboratively on projects, hence creating a virtual learning community.As explained by Chanier (2000) learning methods 'must reflect real social practices' (p.83).Bonk and Cunningham (1998, p. 27) support the same conclusion by stating that learner-centred and constructivist pedagogical formats assist students in constructing their understanding of the culture and communities that surround them.Consequently, learning must be anchored into 'real-world or authentic contexts that make learning meaningful and purposeful'.
Current research in CMC is slowly moving away from quantifiable measurements of success to include the social and affective impact of technology in second language acquisition.Exploring these new directions in CALL research, Kern, Ware and Warschauer (2004) explain that research should be more concerned with 'how well learners can use all their available linguistic, cognitive and social resources to negotiate the linguistic, interactional and cultural demands of online discourse ' (p. 254).This statement is based on the premise that the computer is a pedagogical tool that needs to be carefully used, evaluated and explored by pedagogues and researchers alike.Bates (2000) adds that in order to properly plan and manage new technologies for learning, more research has to concentrate on the application of these technologies to teaching and learning, particularly their relationship to different types of learning.This need for ethnographic and empirical research becomes more pressing if we agree with Warschauer (2002, p. 453) that the computer has already become 'a tool for individual and societal development'.
Research in learning and teaching in higher education is another key area that can have a positive impact onour view of the integration of instructional technology.Based on the assumption that the prime mandate of education is to facilitate and expand students' understanding, Ramsden (2003, p. 8) calls for a 'reflective and inquiring approach as a necessary condition for improving teaching'.Improved teaching translates into improved learning simply because they are inextricably linked.Ramsden's view is that we can enhance university teaching by studying our students' learning.He further explains that, 'this argument rests on the proposition that higher education will benefit if those who teach inquire into the effects of their activities on their students' learning' (p. 7).

Methodology and Analysis
When we started to explore new ways to enhance students' writing experience in French, our first premise was that computers should not simply be considered another pedagogical tool to add to our panoply of instructional resources.As technology is invading nearly every aspect of today's society and our students' daily lives, our responsibility as educators is to inform students of the role and power of technology in the world of written communication.As explained by Biggs (2003, p. 225), our role as educators is to think of technology as not only as informational but also as educational -that is, to use technology as a vehicle for pedagogical change, and not only as a vehicle to transfer or share information faster or more easily.The key is to 'harness technology for more effective teaching'.
Our second premise was based on previous research and findings: that good teaching practice must take into account students' perspectives of their learning and must result in positive learning experiences (Ramsden, 2003;Biggs, 2003).Numerical assessment is not sufficient to reflect a positive learning experience.Rote learning does not mirror the basic human experience of learning, and telling students about principles, facts or ideas represents only one part of teaching.We aim to enhance students' learning experiences through the creation of a virtual place with a focus on inquiry rather than on content.

The Study
To better understand the impact that CMC could have on students in both the cognitive and affective domains, we evaluated a third year French writing course over three semesters.We collected data over 18 months (September 2003 to December 2004) from sources that included participant self-reflections, questionnaires and course artefacts.Our study was not meant to compare a face-to-face environment to an online environment, but to evaluate the impact that technology could have on students' learning and motivation based on their own perspectives.At the same time, we need to realise that this evaluation, and the results it produced, is directly connected to the nature of our teaching.In our case, the constructivist nature of our teaching might have resulted in better outcomes.Such questions and causalities need to be taken into account and properly assessed, as Bates (2000, p. 201) suggests when he states that 'new technologies for teaching do need to be researched and evaluated, but the evaluation should not be restricted merely to replicating classroom learning outcomes'.Bates (2000, p. 202) identifies various research and evaluation questions including one that is directly related to our study, namely: What teaching functions and learning outcomes seem to be more easily or more effectively achieved through the use of this technology?What learning objectives appear difficult to achieve using this technology?Will this technology enable us to achieve learning objectives previously not considered, and are these outcomes worthwhile?
Our evaluation was in part based on the design experiment methodology developed for educational research by Collins (1992) and Brown (1992) and described by Cobb et al. (2003).The main characteristic that we extracted from the methodology is its interventionist nature.As expressed by Cobb et al. (2003, p. 10), 'Design studies are typically test-beds for innovation.The intent is to investigate the possibilities for educational improvement by bringing about new forms of learning in order to study them'.Here, we focused on the pragmatic facet of our study in the hope that further design experiments of the same nature would help develop new theories of learning second languages within the context of CMC.Such theories would contribute to the discovery or description of particular patterns of learning and reasoning in an online setting.
A CMC environment was introduced by way of WebCT into a third year writing class in French.
WebCT was chosen as a course management tool and used to provide students with a shared space, a virtual social forum where they could share ideas, maintain their learning and use additional language tools.Students were organised in groups of 4 or 5 at the beginning of the semester within the online discussion forum.Each week they were given specific activities that involved creating documents to share with fellow students.The assignments were usually centred around students' reflections on an article or points of view on a known concept.In addition to expressing their ideas, students were asked to assess others' contributions to specific aspects of their written work.To encourage time on task, instructions for assignments were posted at regular intervals and student postings had to appear on specific dates.
In this web-based environment, students also had access to reading materials, web pages including interactive exercises, additional class notes, online grammar tools, and other selected learning objects.The activities and supporting documents managed in the online environment were meant to integrate new forms of learning created by students and solicit new learning strategies, in particular meta-cognitive strategies.In addition to the CMC environment of the course, we tried to reinforce meta-cognition in class by ensuring that a process of collaborative and experiential learning was infused into nearly all class activities.
In order to understand learners' perspectives efficiently, we also included a process of evaluation within the course curriculum in order to receive constructive feedback and analyse learners' views of technology and writing.Learner feedback from Semester 1 was the basis from which we implemented the changes for Semester 2. The same process occurred between Semester 2 and Semester 3.During Semesters 1, 2 and 3, data was collected from paper-based surveys and online questionnaires.The course in Semester 4 reflected changes and needs as expressed by students during the previous three semesters.Each new online process was systematically monitored and assessed by users and instructors.The questionnaires with multiple-choice and open-ended questions were used to stimulate a process of reflective practice.The following diagram illustrates how we integrated our foundational theories into our research design: For the purpose of this article we will look more closely at the Fall 2004 semester student surveys, reflections and contributions to WebCT.During this semester, students were given three surveys.All questionnaires consisted of multiple-choice questions, yes/no questions and questions where responses were indicated on a 5-point (Likert) scale with an opportunity to give open comments in some instances.In order to elicit frank opinions, students were advised that no answers would be available to the researchers until all final marks had been sent to the registrar's office.In addition, no identifying data such as names or student numbers were gathered in order to protect the confidentiality of all participants.Data collected from the three questionnaires were analysed both quantitatively and qualitatively and compared to previous data collected (Caws, in press) when appropriate.The number of participants varied from one questionnaire to another, as students were free to answer or not.This also means that some students may have answered Questionnaire 2 and not Questionnaire 3 or vice versa.This variation is to be taken into account and conclusions should not be drawn too hastily.

Course redesign
Original course design

Participants
In the 2004 Fall section a total of 19 students were registered in the course.Three students were male, 16 students were female.Thirteen were aged between 18 and 25, 2 between 26 and 30 and 4 students were over 30 years old.The language experience of the students was somewhat varied: 2 students were French native speakers, 2 considered themselves as having both French and English as native languages, one had French, English and Polish as native languages, one had English and Creole.The 12 remaining students reported having English only as a native language.
Only 8 out of 19 students estimated that their Internet and computer skills were good or very good (that is, about 42%) while 4 students rated their skills as being poor (~21%) and 7 average (~37%).Despite the fairly low average rating in terms of their ability with computers or the Internet, students reported that they did use the Internet frequently, not only outside of school but also for their studies (17 out of 19 students, that is, ~89%).These numbers were consistent all through the three terms and reflect a general trend towards the development of the Internet for educational purposes both in post-secondary institutions and K-12 settings.Students were also asked to rate their level of ability on various computer-related skills.Data shows that emails received the highest rating (3.16) while the use of chat rooms had the lowest rating (1.63).All other skills averaged ratings between 2.10 to 2.68.The level of expertise regarding WebCT was below average at 2.10.These results overall show a mixed level of ability but a fairly consistent level of involvement.

Results
The results are organised around the major themes emphasised in each of the three surveys.A sample of the evaluation instrument is provided.Numerical scores were rounded off to the second decimal place.

Experience with a Course Management System
The following table illustrates the students' experience with a course management system, in this case WebCT.More than half the students (11/19 in the Fall 2004 section and 39/67 in total, that is, approximately 58%) already had experience with such a system although most students reported not having used them as a communication tool.WebCT was the most common course management tool used by students in all sections.When prompted to evaluate the impact of the hypermedia tool on their learning, students were somewhat negative (2.31) at the beginning of the term.One month later, students (N=13) felt that the work they did in WebCT was already having some positive impact on their written language skills (4.38) (Survey 2).This high score was lowered in the last survey (n=11) to 3.88 (survey 3).It is to be noted, with reservation because of the fairly low number of students, that scores increased since the Fall 2003 results (3.07 and 3.03).We can posit that our design experiments model might have had an impact on these numbers since it strives to enhance the course based on students' reflections.

WebCT in the Course
The second survey solicited students' opinions on the use of WebCT in the course, and on the activities and features that they found particularly useful.Students were asked to rate activities on a 5-point Likert scale.Results of survey 2 show an overall positive attitude towards the WebCT component of the course.As previous evaluations revealed (Fall 2003 andSpring 2004), students are particularly keen on having supplementary self-corrected grammar exercises (the online grammar Pomme received 4.18) and on having access to corrections on demand.The collaborative writing exercises received a score of 3.81 and the evaluations provided to students received the highest score at 4.27.In terms of learning strategies and based on the Oxford classification system (Oxford, 1990), these results show a preference for direct strategies: cognitive strategies such as repetition in online grammatical exercises and the practice of writing.Individual comments from students support this view.They commented on the written evaluations (with track changes in Microsoft Word) as a way to help them self-analyse and identify their errors.These comments as well as the scores given to activities where students are engaged on their own in their learning (see tables below) illustrate how indirect strategies (Oxford, 1990) are put into practice when learners understand language education, solicit additional help or seek practice opportunities.In terms of cognitive outcomes, students reported that being systematically evaluated within the context of their original work (the instructor would use track changes to indicate areas where editing was required or advised) helped them understand their errors better because the evaluation was contextualised as opposed to semi-contextualised in the correction summary.Other students admitted they didn't spend a lot of time going over the corrections that they received from the instructor.However, learners felt that their writing skills in French had improved simply because they had spent so much time writing in French, editing their work and peer-editing, hence applying meta-cognitive strategies (Oxford, 1990).
In terms of affective outcomes, some students expressed frustration with regard to the group collaborative writing projects, which they had to do every week.In the same course taught in previous semesters, one student was moderator each week and had to collect writings from each of his/her peers in order to post the final edited paragraph.The idea behind this activity was to put into practice specific cognitive strategies (analysing, summarising, contrasting peer postings) and meta-cognitive skills (organising, self-evaluating and planning) (Oxford, 1990).However, students felt that too much pressure was placed on the moderator who had to wait for peers to complete their tasks.The role of moderator was removed in the last session, which lessened the students' frustrations.

WebCT and Learning and Teaching Practices
In order to better evaluate the value of our educational model in terms of learning and teaching practices, we referred to the principles of good teaching practices developed by Chickering and Gamson (1987).These principles are described as 'encouraging contact between students and faculty, developing reciprocity and cooperation among students, encouraging active learning, giving prompt feedback, emphasising time on task, communicating high expectations, respecting diverse talents and ways of learning'.As noted by Van Gyn (2004, p. 5) 'these principles have proven to be extremely robust and are consistent with current understanding of human learning'.
By infusing a new educational model into a curriculum based on traditional teaching and learning practice, we strove to ensure that our model was congruent with these principles.Consistent with our design experiment methodology, we surveyed students' views on these aspects of the WebCT component as shown in the following table.Results of the last session were fairly consistent with results of the previous year.Wherever students had expressed frustrations with some aspects of the course, a new model was tested, as in the case of the feedback procedure.In previous sessions, only the moderator would receive weekly detailed feedback on language form and the content of submissions.It meant that each student received two detailed feedback reports per term as well as a weekly feedback summary.Comments from the students indicated a need for more regular and consistent feedback.In the last session, detailed feedback using track changes in Word was sent systematically to every student on 6 out of 8 assignments.As indicated in Tables 2 and 3, the scores regarding the feedback received through WebCT are fairly high (4.27 in Questionnaire 2 and 4.08 in Questionnaire 3) and the difference in scoring could be attributed to the specificity of the question (Questionnaire 2 targeting the specific use of track changes for individual correction).
The second highest score in the last session was active learning.Students' additional comments to this effect are quite revealing as to which activities they correlate to active learning (for example, reading and response, compositions, grammar pomme exercises, weekly assignments, pre-reading and activity).Students view active learning as a way to promote their involvement in the course, either individually or though group assignments.In previous surveys (Caws, in press), reading and writing were also considered a good way to promote active learning.Other aspects of the learning and teaching environment in the class are equally important.Social interactions promote indirect learning strategies (Oxford, 1990) and cooperation ensures that the students' motivation is high.A survey done in the other section of the course (without WebCT) in the spring of 2004 (Caws, in press) showed that students felt that the course did not encourage social interaction, nor cooperation among students.Some learners felt that the very high expectations and the focus on memorisation increased the level of competitiveness in the class.These results, although somewhat encouraging for our research, should not lead us to hasty conclusions.Indeed, if students in the section with WebCT generally expressed better satisfaction and motivation towards their learning experience, results at the final exam were similar in both sections.

Findings and Discussions
By engaging students in a process of reflective practice, our assumption was based on a remark by Ramsden (2003, p. 20) that, 'we can only improve the quality of university education if we study its effects on students and look at the experience through their eyes'.In our experiment, the process described above was used as a self-reflection on how the instructional technologies used in the course could help students learn to write quality material in French.
Results of surveys show an overall positive attitude towards the online component of the course.
Students were particularly keen on having supplementary self-corrected grammar exercises online and having access to corrections on demand as shown in Tables 2 and 3.The collaborative writing exercises were considered very useful but very demanding on a cognitive level.This outcome was expressed by many students and was in fact directly related to the purpose of our design.Activities were designed to illustrate cognitive and socio-constructivist theories of learning, and consequently focus was directed towards enabling group collaboration and critical thinking.
Nearly all students felt that their writing skills in French had improved simply because they spent so much time writing in French along with continually self-evaluating their own writing as well as their peers' -hence applying meta-cognitive strategies.These results were consistent throughout the study (Caws, 2005) as shown in students' written feedback and numerical scores.Moreover, by having access to their present and previous documents on demand, as well as to extra material for review and practice, students experienced a sense of accomplishment and felt that they were in control of their learning environment.
The social interaction created by the online activities was considered one of the highlights of the online component.Social interaction was always linked to the course outcomes and recognised as a positive course feature, allowing students to improve their ability to write in French while learning about their peers in a safe environment.This finding is congruent with Oxford and Nyikos' (1989) suggestion that social interactions promote indirect learning strategies and that cooperation among students maintains a high level of motivation.According to the students, the other prominent features of the online component were its promotion of time on task and the encouragement to have contact with the faculty member teaching the course.
Students' somewhat negative initial reaction to the introduction of an online component could be attributed to the fact that, at first, they didn't make any positive correlation between the use of WebCT and the affect on their learning.However we must mention that although more than half of the students already had experience with such a course management tool, most students had never used one as a communication tool to learn specific writing skills.The feeling of frustration expressed by participants is in alignment with the Harasim et al. (1997) study, which shows that students typically express apprehension at the beginning of an online course where the computer is regarded as an inhuman communication tool.To address this issue, I reserved class time in order to explain how some of the learning outcomes could be directly and positively affected by the medium.Despite their apprehension, students appeared to be motivated and keen on learning within a new paradigm.Participation in the activities was high and students enjoyed the sense of community that shaped itself naturally through the online medium.
Interestingly enough, in all three semesters, when we surveyed participants one month after the start of the course, a majority of students felt that the work they did online was already having some positive impact on their written language skills.Students who had initially expressed frustration towards computers agreed that they were learning quickly and were becoming rapidly competent in mastering the 'machine'.In fact, in all three semesters, most students asked to write their final in class essay directly on the computers in order to take advantage of the electronic tools available at their fingertips.

Conclusion and Further Reflections
Studying learning practices through the eyes and minds of our students is a necessary condition to the development of quality teaching in higher education.Research in effective teaching and using technology in higher education is considered a requirement to a better understanding of the value that technology can bring to our curricula.Our study focused mainly on 'design issues' and 'students' responses to e-learning' -two areas considered by Bates and Poole (2003, p. 10) as being critical areas of research.
Despite challenges encountered throughout our study, initial results regarding the integration of new technology for a writing course in a second language are encouraging.By consistently monitoring and assessing our students' perspectives, we can gain a better understanding of how online activities help their learning of key concepts.To better assess the effectiveness of a teaching and or learning technology, Bates (2000, p. 201) states that 'new technologies for teaching do need to be researched and evaluated, but the evaluation should not be restricted merely to replicating classroom learning outcomes'.Our study does not directly address all of the aspects of the evaluation that Bates considered to be key issues; however, it initiates an iterative process by which we can gain a better understanding of what it really means for students to participate in weekly online sharing of original written documents and how this instructional strategy can really help students to learn.
Finally, our study also reinforces the argument that universities must value quality learning in general and quality learning with technology.Only by focusing on this aspect can we enlarge our vision of quality teaching and develop more quality professional strategies to inform instructors of the merits and limitations of using technology in and out of the classroom.

Table 1 :
Students' previous experience with a course management system

Table 2 :
Students' rating of activities in WebCT

Table 3 :
Students' views on learning and teaching practices in WebCT