Designing Ready to Deliver Units of Learning : A Case Study

The field of instructional design and technology has always evolved and grown, translating new knowledge in the learning and cognitive sciences into instructional principles, increasingly incorporating technological innovations into the design of educational solutions, and adapting to social changes (Reiser, 2007; Tennyson, 2005). The ‘learning object’ paradigm, which mainly emerged out of economical and technological concerns, was first developed within the field of software engineering, but is significantly gaining the attention of researchers and practitioners in the field of education at large. Even though this perspective is affecting instructional design practices, still little is known about well-structured methods enabling the support of instructional design processes oriented in this way. We present here a case study, which is part of ongoing doctoral research, and aims to provide a robust instructional design method for the production of learning designs compliant with the IMS LD specification. Our work focuses on the adaptation of an instructional engineering method known as MISA.


Introduction
MISA is a consistent method guiding the design of learning systems.This method makes use of 35 macro and micro design documents (Documentation Elements or DEs).Each DE is composed of attributes, which have assigned values that structure its characteristics in a detailed manner.This pedagogical engineering approach is based on the tradition of instructional design models, the software engineering process, and the field of knowledge management (Paquette, 2002).The MISA method also proposes its own design language (Gibbons & Brewer, 2005) comprising an educational modeling language (Rawlings et al., 2002;Paquette, 2004) and a notation system for the building of the DEs.IMS LD is a leading specification (Koper, 2001(Koper, , 2005) ) within the learning object paradigm, which breaks with the dominant content chunk Learning Object approach.It focuses on modeling activities based on a generic pedagogical metamodel built with EML (Educational Modeling Language), which enables the expression of various pedagogies.IMS LD is of interest to consortiums, researchers, and software developers around the world.Their efforts mainly materialize around applications that enable the representation and interoperability of Units of Learning (UoLs), however it should be emphasized that there is a lack of a robust instructional design method that supports the UoL design process.
While MISA supports a complete instructional design process, IMS LD focuses "on supporting the computational representation of the resources and instruction designed to achieve certain learning, but it is not intended to be used [directly] by final instructional designers" (Caeiro-Rodrígues et al., 2005, p. 2).As expressed by Paquette et al. (2004, p. 5), "the ultimate goal of specifications and standards is to increase interoperability in order to facilitate exchange and re-use of complex systems, and in the case of IMS Learning Design, the exchange and reuse of Units of Learning."Sodhi et al. (2007, p.2) differentiate bottom-up from top-down IMS LD authoring (design) approaches: The authors [designers] can start either from defining the lower process level details and refining the details up, till a learning design emerges (bottom-up), or commencing from selecting the type of education to be modeled and working down to the process level details, aided and guided in the application of learning design rules to capture their knowledge into effective, pedagogically sound UoLs (topdown).Traditionally, strategies for processing information and knowledge ordering, these approaches can also be used to characterize educational process modeling techniques.
In this, we find support for our position that creation of reusable and interoperable IMS LD compliant UoLs is a significant instructional design issue.The proposition of the MISA pedagogical engineering method as a solution for the design of IMS LD UoLs fits well with the top-down approach.The top-down approach is defined as holistic and made concrete through an explicit design process (based on design rules, learning theories, tools and templates, best practices, etc.) that provides sufficient and detailed guidance to the designer.

Research Approach
Our methodological approach is based on a Developmental Research (DR) framework defined by Seels & Rickey (1994, p.127) as "the systematic study of designing, developing and evaluating instructional design programs, processes and products that must meet the criteria of internal consistency and effectiveness." Richey, Klein & Nelson (2004, p. 1102) clearly explain that "while instructional development typically builds on previous research, developmental research attempts to produce the models and principles that guide the design, development, and evaluation processes.As such, doing development and studying development are two different enterprises."These studies involve "the production of knowledge with the ultimate aim of improving the processes of instructional design, development, and evaluation" (op.cit.p. 1099).Van den Akker (1999) analyzes two types of developmental research.Type II focuses on the study of the design and development processes themselves, including tools and models, "in order to come to conclusions concerning design principles of generalizable nature" (p. 6).Richey et al. (op. cit., p. 1103) explain that this second type "is oriented towards a general analysis of design, development, or evaluation processes, addressed either as a whole or in terms of a particular component."The research methodologies in DR propose an array of techniques and tools according to research requirements, organized by "distinct stages, each of which involves reporting and analyzing a data set" (Richey et al, op cit., p.1104).
The first step in our doctoral research was to find valid ways of comparing MISA and IMS LD.From a software development perspective, an ontological comparison (Paquette, 2004) had opened a promising door for the adaptation of the MISA method.This study concluded that the underlying ontologies of MISA and IMS LD share a common perspective as they "put strong emphasis on the representation of pedagogical methods enacted as processes" (op.cit., p.18).Moreover, an exercise in transposition, by an expert researcher, of a MISA compliant instructional scenario into an IMS LD Unit of Learning (De la Teja, 2005) provided new evidence and supported pursuit of the research.This study showed that "MISA is an ID method compatible with the IMSLD specification, because they share a lot of common conceptual elements permitting a harmonious binding" (ibid.p.13).
Based on these works, in our first phase of research, we carried out a different analysis of MISA and IMS LD, from an instructional design perspective, comparing them as design languages (Rheinfrank & Evenson, 1996;Seo & Gibbons, 2003;Gibbons & Brewer, 2005).The results of this analysis helped position MISA and IMS LD with respect to their specificities and boundaries: while IMS LD rests on an educational modeling language or EML (Rawlings et al., 2002;Paquette, 2004), MISA proposes an instructional design language (IDL) comprising its own EML.This EML mainly describes the pedagogical model, one of the four main interrelated models (the other three are the knowledge, media, and delivery models) proposed by the MISA pedagogical engineering method.The MISA IDL can enable a creative process: it has a generative aspect, since "the language can be used as a means of exploring the design space and creating and refining design solutions and alternatives," and a finalist aspect, since "it is used to formalize and "freeze" the final design solution" (Botturi, 2006(Botturi, , p.1219) that can run in a compliant learning management system (LMS).As such, the MISA IDL not only allows the "modeling" of an educational piece (as in IMS LD), but also enables its "design/creation" with regard to four related aspects (knowledge/instruction/media and delivery), through six iterative phases (including the design of the learning solution itself, prior analysis, as well as the planning of its development, implementation, and evaluation).It serves as a conceptual tool (Botturi, op. cit.) for both individual and collective purposes: it supports the designer"s own creative thinking process and the generation of design solutions that can be "communicated" to other designers or stakeholders for discussion and refinement.A notation system (Gibbons & Brewer, 2005) is coupled to MISA IDL for the definition of key DEs involved in the process of designing learning solutions.The MISA EML notation system is called MOT, which stands for "typified object modeling," in French.
There are differences between the IMS LD and MISA EMLs but the aforementioned studies highlighted possible ways of linking them together.Thus, the challenge is to look at the process suggested within MISAand, more particularly, its DEsand find a gradual way of bringing these two modeling languages closer to one another.While supporting a pedagogical engineering process, the EML resulting from an adapted version of MISA should be easily interpretable and translatable into a format that meets IMS LD requirements.
Once the above-mentioned common ground between MISA and IMS LD was identified, thus establishing a base for continuity of the research, we undertook the second phase of our doctoral research where we studied the introduction, into the MISA method, of a new technique 4 supporting the reorganization and representation of a MISA pedagogical scenario according to IMS LD constraints.The aim was to test an "economical" solution that would not require further modifications to the MISA method itself and would lead to the building of an IMS LD compliant UoL.In other words, we explored the possibility of a "shortcut" solution for designing IMS LD UoLs through the MISA pedagogical engineering approach.The assumption that we tested is based on the idea that a "representational technique" could lead to a suitable ad hoc solution.
In this paper, we present a case study corresponding to the second phase of our doctoral research.

The Case Study
Through the case study, we looked for empirical evidence of possible convergence of MISA and IMS LD.We analyzed the transposition of a MISA Instructional Model to an IMS LD UoL representation from the instructional designer"s perspective.
A collaborative (Dillenbourg, 1999;Henri & Lundgren-Cayrol, 2001;Roberts, 2004) and "authentic" (Savery & Duffy, 1995;Duffy & Jonassen, 1991;Reeves, Herrington, & Oliver, 2002) pedagogical model designed for a graduate course in information technology and cognitive development was the starting point.This pedagogical model encourages the learners to participate in an asynchronous virtual scientific conference, the metaphor used to present the four main course activities.As is illustrated in the figure below, these four activities are: preparing for the conference, participating in a poster session, attending a symposium, and participating in the plenary session (Basque, J., Dao, K. & Contamines, J., 2005a, 2005b).deliverables.Our conception of a technique moves away from a simple description of process controls, and towards the heuristic principles that support the execution of different processes.A heuristic principle is not a deterministic rule that prescribes the proper way to proceed and absolutely guarantees success.Rather, it provides advice that will generally allow those who heed it to obtain satisfactory results (Paquette, 2002b, p. 111).
We use a concrete developmental research methodology (van den Akker, 1999; Rickey, Klein and Wayne, 2003) called "formative research" (Reigeluth & Frick, 1999, p.633) that is "intended to improve design theory (or models) for designing instructional practices or processes."The authors just quoted explain, based on overwhelming evidence, how formative research methodology is useful and appropriate to improve theories and models in almost all fields of education.This methodology follows Yin"s (2003) four-stage case study recommendations: design the case study, conduct the case study, analyze the case study evidence, and develop the conclusions and recommendations.

Design of the Case Study
In our case study, we focused on two main aspects: (1) clear identification of MISA elements and processes to be modified and (2) verification of the appropriateness of the principles guiding the MISA ID process with regard to the design of a UoL.
The case is defined as the transposition of a MISA pedagogical scenario to an IMS LD compliant UoL, through a representation technique (the editor"s User Guide), from an instructional designer"s perspective.The technique is a procedure that the designer must apply to reinterpret (translate) and rebuild a MISA pedagogical model into an IMS LD UoL syntax.Based on the evidence establishing that the EMLs of MISA and IMS LD are similar but not identical, the technique represents an ad hoc solution for the reorganization of the scenario.If our assumptions are correct, the representation technique is not sufficient to succeed in building a UoL and changes to the MISA method itself are necessary.
During the exercise the participant was allowed to use the following artifacts: The MOT+ LD editor: the software application that offers a graphical way of representing a UoL.The MOT+ LD editor evolved from the MOT editor used in MISA for the creation of the knowledge, instructional, media and delivery models.
A narrative template: an IMS LD structured Narrative that allows collecting the required information for an IMS LD Level A to be represented in the editor.
The MOT+ LD editor"s User Guide: a representation technique that allows representing a UoL within an IMS LD (Level A) framework, using the MOT+ LD editor.IMS LD reference documentation: documents with information about concepts related to the specification, which help the participant understand the task to be accomplished.

Conduct the Case Study
The participant was an instructional designer and cognitive modeling expert with 12 years of experience.He also had 7 years of expertise using the MISA method and 10 years using various versions of MOT5 software.He had designed 4 full-fledged online courses applying MISA and MOT, and had also worked as an online course facilitator.Moreover, he had very little prior knowledge of IMS LD.
The case study sessions consisted of a half-hour introductory session and two subsequent threehour work sessions.Sessions took place at the LORIT6 , a distance learning research laboratory at LICEF/Télé-université.
During each session, we gathered data using the LORIT"s equipment and services.We recorded the designer"s work environment (from three different angles) and the video screen signal from the computer in order to keep track of the designer"s use of the modeling software tool.We also employed a think aloud protocol (Ericsson & Simon, 1980, 1993) and recorded the designer"s verbalizations and explanations of the ongoing activity.This data was supplemented with notes from the observation of important events that we identified.After the end of each session, we kept copies of the designer"s work in progress (i.e., files with the different stages of the MISA pedagogical model, reorganized as a UoL in progress).Each session was concluded with a debriefing.After the last session, an interview with the designer took place.
Of the four preplanned sessions, only two actually took place.After the second one, we decided to end the case study, given the participant"s rapidly increasing need for assistance.The session was becoming more and more like a tutorial, thus moving away from our objectives.We also considered that we had collected enough information for our purposes.

Analyse the Case Study Evidence
We have divided the analysis of data into two sections: (1) a comparison of the MISA Documentation Elements (previously produced by the participant) with the documents created by the participant as a result of the sessions; and (2) the analysis of the UoL representation process.
The purpose of the first section was to identify, within the MISA DEs and the course itself, the attributes and values that were reused to represent the UoL.We were careful to note which DE elements were consulted by the participant during the sessions We then proceeded to conduct deeper analysis, so as to be able to later compare these elements with the documents resulting from the sessions.
Based on this analysis, we identified syntactic and semantic correspondences and non correspondences between the elements describing the "two types of scenarios", i.e. the elements from the sessions" outcomes and those from the MISA DEs previously provided by the participant.The identification of DE attributes and values is not sufficient in itself to isolate all the elements that are common to MISA and IMS LD.How they are organized and structured and how decisions are undertaken must also be examined.We explored these questions through process analysis, a dynamic view, which is complementary to the rather static analysis of the artifacts produced by the participant, based on the case scenario.
The purpose of the second section (analysis of the representational process) was to identify critical elements that can provide guidelines, in regards to the MISA design process, leading to the modeling of a UoL.
In order to reconstruct the participant"s activity, we created a table clearly differentiating the prescribed tasks from the activity actually carried out by the participant.The "reconstruction" of the participant's procedure is based on the information gathered through the video and audio recordings, observation notes, debriefings, and final interview.

Develop the conclusions and recommendations
The participant represented an incomplete and only partially compliant UoL.At first glance, it might be believed that a UoL was actually built during the sessions.However, deeper analysis of the work and materials produced by the participant allowed us to realize that this result was not an IMS LD-compliant UoL, even if most of its components could be identified.
The sessions revealed many key strategic issues that point towards possible modifications to MISA.First, they allowed us to identify the MISA Documentation Elements that are directly involved in the representation of a UoL.Consequently, this has allowed us to identify other DEs that were used beforehand in order to create those directly involved in the modeling of a UoL.Next, we were able to explore the limits of the theatrical metaphor as a framework for thinking and designing a learning sequence.Also, we identified elements in the MISA and IMS LD languages that interfere with each other, due to the presence of identical or similar terms with different (or slightly different) meanings.Finally, the case study has served to highlight that a representation technique, as an ad-hoc "solution", is insufficient for the creation of a UOL.It is essential to provide the designer with a robust and complete method of instructional design.

(Re)use of MISA Documentation Elements and attributes
During the sessions, the reuse of Documentation Elements and attributes to produce a UoL was evidenced.Learner/support activities and resources, together with some of their attributes, are reused during scenario transposition-but here is what our analysis highlights: -The attributes from the MISA DEs related to the "instructional model" are reused but are organized differently.While IMS LD presupposes a strict way of structuring learner and support activities together with environments (composed of learning objects and services), focusing the learning flow on delivery (or run), the MISA pedagogical model is more flexible with regard to the way in which the learner and support scenarios are built, and focuses, rather, on instruction.When designing the pedagogical model, MISA focuses on the organization of learning events and activities that meet the curriculum requirements and the guidelines of a chosen pedagogical approach.In MISA, the constraints of delivery and execution are discussed later, when designing the Delivery Model.
-MISA proposes (although does not require) the declaration of instructional rules (viz.study approach, collaboration, evaluation, and customization) that are statements guiding the completion of the learning events, the learning units or the learning activities in the instructional scenario.The MISA DEs provided by the participant were poor with respect to declaration of rules.We suppose that the explicit declaration of rules will enable the reorganizing of instructional scenarios, during the building of the Delivery Model, according to IMS LD restrictions (boundaries).Special attention must thus be given to the DEs intended for explanation of the activity attributes.
-The MISA resource taxonomy includes several different types of educational materials as well as editing and communication tools.However the MISA DEs currently do not allow the classification of resources as Learning Objects and Services, just as in IMS LD.The simple addition of an attribute to MISA DEs could allow the classification of resources according to the distinction made in IMS LD.In this sense, we should explore adding attributes to the MISA DE presenting the "list of learning materials."Consequently, the creation of IMS LD environments (which gathers LOs and Services together) within a MISA pedagogical model can in turn be facilitated.

Criteria for breaking down the UoL
The case study showed that the concepts and structure of MISA and IMS LD share common ground but that this is not sufficient to ensure the production of a coherent UoL.Correspondences between the MISA Instructional Model and the UoL are not one to one.We must explore ways to establish criteria for breaking down instructional scenarios.MISA deploys an instructional structure, which is a structure of learning events that shapes the curriculum/syllabus-related hierarchy (program, course, module, lessons, chapter, unit, etc.).This structure has attached instructional scenarios that articulate the learner/support activity flow, including the required resources.The entire structure comprises the instructional model, whose granularity rests upon the designer"s criteria and contextual constraints.This "model" break down addresses a semantically and educationally grounded way of decomposing the learning flow.In IMS LD the concepts structuring a UoL are of generic interest.This structure is based on the theatrical metaphor that shapes EML encoding according to run-time requirements.It is proposed as a metascenario or, better, a metamodel.The level of refinement of a detailed UoL run-time description usually exceeds that of a MISA learning scenario.However, in MISA the run-time or delivery aspects of a learning system are addressed when constructing the Delivery Model.The MISA Delivery Model describes the roles of the actors during the delivery of a learning system as well as their interactions with the course structure, materials, tools, means of communication, services, and locations, which they either use or supply to other actors.In this sense, it completes the learning scenario, focusing on delivery matters.We must thus examine the tight relationship between the MISA instructional and delivery models.We must also look at the possible impact of IMS LD Levels B (which stipulates additional conditions for progression within the learning scenario) and C (which triggers notifications as a form of event-driven messaging sent both to elements of the design and to human participants), especially from the perspective of rule declaration, which we previously mentioned.We must also enrich the IMS LD metaphor with more learning specific terminology in order to reduce the level of abstraction of the metamodel.Instead of compelling the designer to create a UoL in one single operation, we advocate for a gradual creation of a UoL based on the MISA Instructional and Delivery models.

Terminology
The editor"s User Guide employed by the participant during the case study used IMS LD"s terminology, which diverges from the instructional design terminology proposed by MISA.Identical terms with different meanings in MISA and IMS LD caused misinterpretations and induced errors.The MISA design language is more appropriate for the designer and only requires minor modifications or additions to achieve a solution for certain problems identified during the case study sessions.For example, "environments" in IMS LD are containers for specific learning objects and services required for the accomplishment of a given activity; this is similar to the notion of "package" in the MISA Delivery Model.The notion of IMS LD "learning objects" and "services" is encompassed by the concept of "resources" in MISA.

Design process
The case study revealed that building a UoL from scratch is an arduous and complex enterprise.Designing and modeling seem to be different in nature.While modeling (such as in IMS LD) focuses on the "shape" and "compliant arrangement of elements" of an educational piece, designing (as in MISA) encompasses a progressive and iterative process of thinking about, generating, creating, and adjusting learning situations.In other words, while MISA supports a multi-layered problem solving approach to the design of a learning solution, IMS LD is focused on achieving the right arrangement of learning scenario elements, apt for execution by a machine.In this sense, an IMS LD UoL can be understood as "a result" of the instructional design process, like a snapshot of a very detailed instructional scenario set up for delivery.Representing a UoL thus presupposes a previous process of instructional design where crucial decisions (about knowledge to be assimilated, learning objectives, target learning profiles, learning events, learner and staff activities, pedagogical materials and services, etc.) have already been undertaken.

Future research
We have conducted a case study where a technique for the representation of IMS LD UoLs was applied to the transposition of a MISA pedagogical scenario by an expert instructional designer.The results of this phase of our research revealed that such a solution is insufficient, but gave us in-depth information about how to revise MISA principles, MISA DEs, and the MISA language.It also provided knowledge about the main steps of a UoL design process using the MISA approach, and more detailed information on terminology similarities, equivalences, and overlapping.
This case study mostly showed the importance of maintaining the MISA process and structure unchanged.We can also conclude that there is a need to modify certain Documentation Elements that are fundamental for keeping track of the whole design process.We have also identified the need to expand MISA rule declaration to facilitate UoL break-down as well as the need to add attributes to appropriate Documentation Elements in order to describe IMS LD activities, resources, and sequencing.This case study let us identify most of the MISA Documentation Elements and attributes needed to describe an IMS LD UoL as well as instructional design principles for an adaptation of MISA to the design of a reusable instructional model (or scenario) based on the IMS LD specification.It also allowed us to collect empirical evidence indicating the need for an adaptation of MISA to produce compliant Learning Designs.
From the MISA perspective, a UoL is the result of a pedagogical engineering process.In other words, it is the outcome of a problem solving and decision making process of learning design.The desire to adopt approaches related to the design of learning solutions in order to make them interoperable and reusable adds another layer of complexity to an already intricate process of instructional design.
The next steps of this doctoral research are the introduction of modifications to MISA based on this case study"s results, the validation of the new version of MISA using a two-round Delphi expert validation technique and, finally, the introduction of further modifications to MISA resulting from said expert validation.
The results already obtained let us foresee that, in addition to modifications to the MISA DEs, the following related matters will also have to be explored: improving the software tool to cope with the designer"s activity and redefining designer competencies that are required to achieve an IMS LD compliant UoL, while striking a balance between these two endeavours.

Figure 2 .
Figure 2. A MOT+ LD editor window MISA Documentation Elements: macro and micro design documents from a MISA compliant course (that had been designed, previously, by the participant), together with the Course Web site itself.

Figure 3 .
Figure 3.A section of a MISA Pedagogical Model represented with MOT

Figure 5 .
Figure 5.An excerpt from the table used to analyze the process of representing a UoL with MOT+ LD