Dynamics of Instructional and Perceptual Factors in Instructional Design Competence Development

The nine-month study reported in this paper used mixed methods data and a qualitative analysis to examine the skill and perceptual development of 17 graduate design students. Individual differences, perceptions and preferences that apparently promoted rapid and productive development included: design efficacy, mastery goals, preference for cognitive challenge and tolerance for risk-taking. Novice learners benefited from content and context familiarity, but as they developed competence, they gained from choosing less familiar tasks and content. Features of the learning environment identified as contributing to novice designers’ knowledge and skill development were: authentic projects, detailed assignment specifications, multiple types and levels of feedback, and clear alignment with professional performance standards. These findings inform the strategic design of instructional opportunities for novice designers and similarly complex applied professional fields.


Introduction
Instructional design (ID) competence has alternately been presented as an art, craft, or scientific process or as some combination of these (e.g., Clark & Estes, 1998;Gibbons, 2003;Merrill, 2007;Parrish, 2005;Rowland, 1991;Smith & Boling, 2009).Similar to engineering, architecture and other design fields, ID calls on a complex applied skill set that must be used adaptively in everchanging contexts of professional practice (Hardré, 2003).In such a diverse field with myriad applications, how can foundational coursework in ID effectively prepare students for professional practice?This paper examines the effects of a subset of instructional, perceptual and environmental factors on the development of ID competence.The educational implications of these factors highlight the need to consider their interdependent relationships, rather than treating them as independent of each other.
ID is "increasingly influential in job markets (Atal, 2007;Hempel, 2006) and in academic circles (Woyke, 2007)" (Smith & Boling, 2009, p. 4).The field of ID has been challenged to acknowledge a broader set of frameworks that underscore the social, emotional, aesthetic and organisational complexity of authentic human interactions (Grunert, 1997;Norman, 2004).Even with the increase in exposure to the skill of ID, little research has examined the influence of instructional, perceptual and environmental features within design courses.This study investigated

Development of Expert and Novice Practice
The nature of expertise across applied fields, such as design, has been framed both in abstract definition and in terms of the practice of expert professionals.Expertise includes extensive knowledge, but also extends to application of that knowledge, in strategic skill that is easier and more fluid for experts than for novices (Anderson, 2000;Dreyfus & Dreyfus, 2005).Expertise has been theorised on a number of developmental continua, with various stages and sets of labels (e.g., Chi, 2006;Cross, 2007;Dreyfus & Dreyfus, 2005).Some definitions of expertise hold up doing different and more challenging tasks well as the demonstration of greater expertise than doing simpler tasks (Bereiter & Scardamalia, 1993), often including elements of adaptability that lead to potential for innovation (Hatano, 1990).Other definitions of expertise posit that doing the same tasks more quickly (automatically) demonstrates greater expertise than doing them more slowly (deliberately) (Anderson, 2000).One estimate is that an individual requires 10 years and 10,000 hours of explicit and deliberate practice to become expert in an applied professional skill (Ericsson, Krampe & Tesch-Römer, 1993).However, even over time, practice must increase in complexity and challenge or it can result in little to no change in true skill (Fadde, 2009).Moving learners toward greater expertise, or competence attainment in a complex, applied field, requires providing clear performance standards, with continuous practice, and standards-attentive feedback on their developing expertise (Fadde, 2009).The schematic organisation which experts develop is highly situated, dependent on its strategic context-of-application (Ross, Shafer & Klein, 2006), and it includes multiple potential strategies to solve problems (Anderson, 2000;Bereiter & Scardamalia, 1993;Bransford, Brown & Cocking, 1999;Goel & Pirolli, 1992).
In ID specifically, experts have more complex thinking, more systematic approaches to problem representation, and more iterative reflection and monitoring than novices (Perez & Emery, 1995).Competent designers use data to inform and support analysis, design and evaluation decisions (Dick, Carey & Carey, 2009;Piskurich, 2006;Richey, Fields & Foxon, 2001).In contrast, novice designers tend to fail at making informed design decisions and focus on producing flashy instruction without considering the implications to learning (e.g., use of modalities, distraction) (Alessi & Trollip, 2001).Traditionally, ID has been presented as a strategic problem-solving task (Richey, Fields & Foxon, 2001;Morrison, Ross & Kemp, 2007).With its emphasis on a replicable, systematic process (Brown & Green, 2006;Cennamo & Kalk, 2004), the ID literature is relatively silent on the role of the individual designer (Smith & Boling, 2009).However, individual differences in philosophy, background, perspective and experience could account for variance of interpretations, application, precision and creativity in design (Owen, 2007).With increased attention to creativity, aesthetics and fine points of judgment (Molenda & Boling, 2008;Rowland, 1995), it becomes more important to investigate the developing practice of today's novices who will become tomorrow's designers (Rowland, 2008).With this forward-looking perspective, an element critical to innovative design competence is the development and nurturing of designers' independent judgment and range of application potential (Cross, 2007;Nelson & Stolterman, 2003;Schön, 1987).

ID Learning and Skill Development
While the practice of expert designers is given much research attention, far less investigation has been done on the development from novice to expert (Dorst & Reyman, 2004).A few studies have investigated learning and development of ID knowledge and skill among students of ID (Ertmer & Cennamo, 1995;Ge & Hardré, 2010;Hardré, Ge & Thomas, 2007;Quinn, 1994), and of designers in specialised contexts (e.g., Christensen & Ogusthorpe, 2004;Rowland, 1991).Some research has also been done on graduate teaching assistants learning ID as a strategic skill for transfer to their own professional teaching practice (Hardré & Chen, 2005, 2006).Yet there exists comparatively little understanding of skill development, adoption, and transformation of practice (Dorst & Reyman, 2004).While it makes sense intuitively that project-based instruction is best for ID skills, little research exists that explains how and why that pedagogical approach works.

Collaboration and Social Learning
The social environment plays a significant role in learning and development, as learners experience peer and personal success and build personal efficacy for new knowledge and skill areas (Bandura, 1997).Teaching and explicitly modeling expertise in any field requires the expert to make the invisible visible, to make implicit internal thinking processes explicit and apparent to learners (Fadde, 2009).Collaborative and team-based learning activities deepen intellectual development as individuals challenge one another's perspectives, through mutual inquiry and reflection (Lawrence, 2002).This type of learning environment provides multi-source competence feedback that stimulates metacognitive awareness and self-monitoring.Through giving and receiving feedback, learners practice the application of cognitive and affective references to the professional standards and principles of practice.Dynamic group engagement creates social spaces that facilitate inquiry, linking learners and instructors as partners in unique knowledge creation (York, 2005).In addition to the role of collaboration in learning design, collaboration is a critical skill in professional design practice (Cennamo & Kalk, 2004;Fadde, 2009;Richey, Fields & Foxon, 2001), so it is a component of design expertise.

Project-Based Instruction
While general learning theory informs characteristics of project-based instruction for practice (e.g., authenticity, alignment with target performance standards), numerous questions remain that could inform design specifically for learning in design.What elements of project-based instruction and activities most effectively support ID skill development, and for what types of learners?What constitutes the best balance of various features of instruction in project-based design, such as: support and autonomy, constraint and choice, experimentation and scaffolding, and selfmonitoring with feedback and coaching?How does that balance shift for different groups of learners, types of design goals, and contexts-of-use?Given the complexity of ID for new learners, and the interaction of design and content knowledge, does content familiarity enable or constrain the development of novices' design skills?Are there individual difference characteristics that interact with various environmental features in ways that support or thwart the development of design competence?Given the various definitions of, and foundations for, design competence, do the answers to these previous questions depend on how we judge design quality, or do they converge?50 years and the initial group included 10 females (60%) and 7 males (40%).Ethnicity of participants included: Anglo/White (n=10), African American/Black (n=1), Latino/Hispanic (n=1), and Asian/Chinese/Korean (n=5).The participants' majors and status ranged from first-year ID master's students to doctoral students in fine arts and psychology.However, all of the participants were characterised (by themselves and the researchers) as novice designers, and all sought to become competent in ID, to promote achievement of their career and professional goals.

Course Context and Structure
The two sequential courses (ID I & II) are first-year graduate courses in ID.Their scope is process, principles and application of ID, including opportunities to experience the systematic process of designing and developing replicable instruction.The foundational content for the course was in three textbooks (Dick, Carey & Carey, 2009;Morrison, Ross & Kemp, 2007, Smith & Ragan, 2005), among the most used and recommended texts for university-based ID courses (Reiser, Mackal & Sachs, 2005).Activities were grounded in adult learning theory and instructional principles (e.g., Gagné & Medsker, 1996;Knowles, Holton & Swanson, 1998;Merriam, Caffarella & Baumgartner, 2006).
The project-based courses provide students with guided questions and incremental design tasks (i.e., project prospectus, needs analysis, learner analysis, task analysis, instructional plan and evaluation) to assist them in applying principles and developing adaptive competence in application and alignment.In addition to individual ID projects, students also collaboratively engaged in analyses of professional designs and case studies.These group projects required students to collaboratively identify complex problems, share multiple perspectives, and negotiate final strategies.
Productivity demands increased over the two-semester sequence.In Semester One, students produced one complete design packet, any content, media and learner group.In Semester Two, students produced four complete design packets, with required changes in content, media, context and learner group across the four designs.The four design assignments also presented increasing complexity in their target instructional outcomes.This assignment structure required both increased complexity and increased speed of design performance to demonstrate competence development.Students received three types of feedback on each design document: (i) peer critique and feedback from two to four classmates; (ii) general feedback from the instructor to the class on common errors made on the assignments (given out in the class notes and discussed in class); and, (iii) individualised feedback from the instructor on the student's specific design work (1-2 pages, handwritten, private).
Both courses required students to provide rationales (expose their design reasoning) to justify their strategic design choices.They also required metacognitive reflections, challenging learners to thoughtfully self-evaluate (in light of feedback from the instructor and peers), and to track their own knowledge, skill and competence development over the 9-month study period.

Data Collection Procedures
Students were invited to participate by the second researcher (not the instructor), and all agreed.Questionnaire instruments were administered online through the SurveyMonkey © secure digital administration system.They were completed during class sessions scheduled a priori in the study design, with administration supervised by the second researcher.Students submitted assignments for the research analysis privately on digital media.All instruments and activities were approved by the institutional review board and all followed human subject protection protocols.Participant confidentiality was protected throughout the study.
Mixed-method data collection included in-class observations, questionnaires, student assignments, and structured interviews.Data was collected across two semesters to track growth and development of ID skills and perceptions of competence in ID, in addition to identifying and verifying apparent influences on that development.Assignments and data collection activities and sources for ID I and II are shown in Table 1.(Ge & Hardré, 2010;Hardré, Ge & Thomas, 2007).The observation protocols were unique to this study, but like all of the other instruments they were designed by grounding in the relevant theoretical literatures and reviewed by multiple experts before use, providing confidence of their validity for use in this study.
Goals.The instrument measuring goals was the Approaches to Learning (ATL), which identifies learners' personal goal structures (Greene & Miller, 1996).Sample items by subscales include: learning/mastery (i.e., "I do my work in this class because I like to understand what I am learning"), performance-approach (i.e."I do my work in this class because I like to score higher than the other students"), and performance-avoidance (i.e."I don't do my work in this class so I can avoid looking stupid to others").
Need for structure.The 12-item Need for Structure (NFS) scale assesses learners' preference for well-structured events versus tolerance for work that is less structured or uncertain (Neuberg & Newsom, 1993).Sample item: "I don't like situations that are uncertain".
Need for cognition.The 18-item Need for Cognition (NFC) scale assesses preference for deep thinking and intellectual challenges (Cacioppo, Petty & Kao, 1984).Sample item: "I find satisfaction in deliberating hard and long for hours." Perceived design competence.The 3-item Perceived Design Competence (PDC) instrument assesses self-perceptions of competence in design (Hardré, 2003).It presents a general stem, with three quality descriptors (capable, achieving, and competent).
Design efficacy.The 8-item Design Efficacy Scale (DES) assesses learners' perceived capability of succeeding at design tasks even when faced with challenges (Hardré, 2003).Sample item: "If a task analysis shows that the task is especially complex, I believe that I can handle the design challenge." Satisfaction with design knowledge and strategies.Three sets of parallel 4-item perceptions subscales assess design students' personal satisfaction with the adequacy of their design knowledge and strategies to meet the needs that confront them (IDS; Hardré & Chen, 2005, 2006).Sample item: "I am satisfied with my current knowledge of how to design instruction."

Assessment of ID knowledge and strategies.
A set of open-ended qualitative response instruments (6 items), assessed knowledge of basic components of ID process, and instructional design principles (IDK; Hardré & Chen, 2005, 2006).Sample item: "What do you need to consider to design a learning environment in which students can learn successfully?"

Reflection on environment and development.
A set of open-ended response items directly addressed learners' perceptions of the effects of the class learning environment on their own knowledge and skill development in ID (end of each semester).Sample item: "What was the specific role of the various peer activities in your development (e.g., case analysis team activity, peer feedback on assignments, team projects)?" Observations.Two independent observers recorded observations of students, individually and in groups, across a range of behaviors and characteristics.Both sets of observations produced data in quantitative and qualitative forms.Three times each semester, the instructor recorded observations of students' in-class design activities.Dates were set a priori based on the class schedule.Three times each semester, the external researcher-designer made random visits for systematic observations of learners' design activities.The researchers used an observation protocol that ensured consistency across users and instances.

Student work
The student work is a collection of assignments completed throughout the two-semester course sequence, along with peer and instructor critiques and feedback.The performance standard was a design that enabled either accurate delivery by an appropriately skilled instructor, or accurate production by a development team.Student work was a collective and developmental indicator of learners' development toward ID competence.It was evaluated using rubrics based on a range of ID paradigms (e.g., Kruger & Cross, 2006;Perez & Emery, 1995;Richey, Fields & Foxon, 2001).
Individual and group design products.These designs were scored by two independent raters, using original product scoring rubrics developed from a range of sources of evaluation of design quality: 1) the IBSTPI ID Professional Competencies (essential) and aligned with Dreyfus's first five levels of expertise development (novice to expert); and 2) a set of qualitative descriptive characteristics from the ID professional literature.The assignments for each semester are listed below.
Semester 1 project.Students completed one complete ID packet, from analysis through design and plan of evaluation, submitted as a set of five incremental documents (three analyses, design specifications, evaluation plan), and then revised for a final submission.
Semester 2 projects.Students completed four complete ID projects, each producing a different type of primary learning outcome in ascending complexity over the semester: lesson 1 (declarative or concept), lesson 2 (principle or procedure), lesson 3 (cognitive strategy or problem solving), and lesson 4 (motivation, affect or psychomotor.) Metacognitive essays.A reflective, self-evaluative essay was submitted with the revised projects, at the end of each semester, encompassing: self-perception of ID knowledge and performance, learning and skill development, how development was demonstrated in design project(s); goals and strategies for continued expertise development.
Weekly "Active Reading" responses.These weekly writing assignments were in response to the assignment to "engage and apply the content from the readings.Don't just summarise" (1-2 pages, typed).
Collaborative tasks.Collaboration and team tasks over the semester include peer critique and feedback on design documents at drafts and final stages; and analysis of ID readings, case studies, and professional projects.

Interviews
Interviews were conducted using a semi-structured interview protocol, covering: personal learning experience, instructional design perspective, perspective on reflections, and class learning environment.Interviews were used to confirm and elaborate the other data sources, on apparent patterns of skill development and experience of the critique-and-feedback process.Sample questions: "What are some of the assignments and projects that you think are particularly useful to help you develop expertise in instructional design?";"Please describe your experience with the peer review and feedback in this class."and "How would you describe yourself as an instructional designer?"

Data Organisation and Analysis
The interviews and handwritten notes were transcribed by the College's research centre staff.Data was organised as participant cases, in case documents (Excel spreadsheets and Word files).Data analysis was qualitative in process, progressing through phases of: coding, cross-referencing, comparison and triangulation, and synthesis into patterns and themes (Saldaňa, 2009;Yin, 2003).
The two researchers independently scored the design products, ran descriptive statistical tests as appropriate on quantitative indicators, and coded the qualitative information, then joined together for synthesis.A cause or influence assertion depended on multiple sources of data (e.g., quantitative and qualitative, self-report and observer) that converged to support more than a simple correlative relationship among factors (Yin, 2003).
Coding.The two researchers initially coded the data independently from the case files, and also listened to the original audio recordings to include nuances of emphasis and language.Then the two researchers met to compare their notes and coding and collaboratively merge them into preliminary case files.They iteratively returned to the raw data as needed.
Cross-referencing and triangulation.The triangulation of data was conducted across three dimensions: (a) case-level data triangulation, (b) change-trajectory cross-referencing, and (c) casecomparison, multi-investigator triangulation.The researchers analysed all the data in parallel for each case to seek the trajectory of changes across the two-semester study.At the case level dimension, each individual participant's case data profile was examined by comparing different data sources collected within a given time frame.For example, the first semester's design documents and metacognitive essays were compared with the first semester's interviews, peer feedback, and observation notes.At the change trajectory dimension, each individual case was examined by type of data and sources, compared over time through the nine-month study period.
For instance, individuals' interviews conducted at different points of time were compared, and the first semester's design packet was compared with the second semester's design packets.At the multi-case, multi-rater dimension, different types of data of an individual case were also crossreferenced and compared with other cases.Collaborative notes and results of the researchers' observations, interpretations, and reflections, resulting from the multiple dimensions of analysis, were distilled to address the research questions.Results are organised by themes and patterns relevant to the research purpose.

Results
In seeking to uncover individual difference characteristics that appear to influence students' skill and competence development, we found that several types of individual differences appeared to be related to (and may have contributed to) design students' skill and competence development.Some of these were matters of explicit choice, some developed over time, and others were more stable differences of perception or personality.These influential differences included: content and context familiarity (e.g., choosing less familiar content areas), choice of range in design tasks and content areas (e.g., choosing more complex and difficult design tasks), and a cluster of quantifiable characteristics of questionnaires (design self-efficacy, mastery goals, preference for cognitive challenge, and tolerance for uncertainty and risk-taking).
Early, their content area familiarity and context-related background helped to scaffold them through the design process, as depending on content familiarity in early designs freed up their attention and energy for learning the ID content and principles.However, later in their development, those who continued to design in content and contexts of familiarity demonstrated less rapid ID skill development, not experiencing the same range of varied practice experiences and challenge as those who chose content and contexts that were unfamiliar.General background and skills also supported the development of some specialised ID competencies over others.For example, students with math, science and logic background found analysis skills easier to develop, while those with K-12 teaching background tended to skip analysis and move straight to designing instructional strategy (and had to build backward to develop analytical reasoning).Those with IT background gravitated toward design of media and materials, but found analysis, assessment, and rationales more difficult.The challenge in developing well-rounded ID competence was to stretch into those ID skills and competencies least supported by background.They did this by taking advantage of resources in several ways: (i) by learning from peers' advice and examples in their areas of strength; (ii) by "digging into" the texts and resources for principles and strategies (they read most deeply the text sections on their weakest areas); (iii) by going to the instructor for additional explanation, advice and resources to strengthen their skills; or (iv) by seeking out expert resources and examples (such as from the online resource list provided for the course, or from their own searches).
Another important difference was challenge-seeking, or the degree to which learners chose to engage in original design (from analysis up) or some partial design task (such as translating existing instruction from face-to-face to e-learning, or updating existing content).The full range of design tasks and translations were allowed and supported, as equally authentic tasks.However, they differed in the degree to which learners had to engage in original work (vs.depending on work done by others).Like the difference in familiar and unfamiliar content areas, the range of design tasks that a learner stretched to take on influenced development over the two semesters, through diversity and challenge in practice opportunities (also consistent with Bereiter & Scardamalia, 1993).Some students had higher efficacy for challenge, more willingness to take risks, and more receptivity to experiential learning (expressed in their reflective and strategic writing over the study period), and these characteristics seemed to support those who reached higher and farther.Those who took safer, smaller tasks, with less original analysis and design work required of them did not develop as rapidly as their peers who tried more original, complex and difficult design tasks.
Self-efficacy for the work of design was linked to knowledge and skill development.This characteristic developed markedly over time, consistent with spaced practice and incremental success on increasingly complex design tasks.Self-efficacy was also related to learners' desire to engage deeply in the cognitive challenge of learning new skills, and their willingness to take risks by trying to design in unfamiliar content and context areas, rather than keeping close to the familiar.Initial design self-efficacy was related to learners' prior experience in, or familiarity with, ID, but over time it developed rapidly in response to current knowledge and experience.
Goal orientations stood out as an important individual difference that influenced design development.Those who came with performance goals tended to resist the developmental focus initially, and simply wanted "a template" a quick, easy formulaic approach into which to insert content, not the requisite of developing adaptive skills.Their early assignments tended to be very formulaic, and they expected high grades for filling in the blanks, treating the specifications as checklists.Their work tended to be technically sound, but cursory rather than thoughtful, wellanalyzed and well-developed.Those with mastery goals were more likely to ask questions about how to apply the principles and address the specifications.Their early assignments tended to be less perfect in addressing the specifications, contained more departures from the norm, more development of key ideas (even resulting in longer documents as they "chased" ideas).
Learners with higher preference for cognitive challenge, tolerance for error and uncertainty, and willingness to take risks tended to develop a wider range of design skills and strategies more quickly, and finished the course(s) with relatively more skill development than their more conservative, risk-averse peers.As indicated above, this cluster of characteristics was also related to both mastery goals and design self-efficacy.The instructor promoted mastery goals, tolerance for error, uncertainty and risk-taking, throughout the two-semester course sequence, through: modeling mastery development, sharing stories of learning from errors, encouraging risk-taking in assignments, active coaching, and individualised feedback.
Design self-regulation (as task management, self-evaluation and monitoring), increased with knowledge and skill, and was reflected in both peer critiques and responses to critiques (notably in clarity of points and rationales).Individual differences noted above also supported the development of self-regulation.Goals and other productive preferences were more stable with mature, self-directed (self-aware and self-regulated) learners.Notably early on (Weeks 4-8) there was a shift toward mastery goals, and then another forward shift later (Weeks 26-28).

Limitations
With six of the 17 students not continuing into ID II, we have only limited data for them.The study would be strengthened by having all 17 participants for the full study period.

Conclusion
Our findings are complex, consistent with the nature of ID, the dynamic of active learning in an applied classroom with multi-source inputs and authentic project-based tasks.They might be summarised by considering the way that assets to development and evidence of that development emerged.
First, overall, the role of individual differences interacting with the learning environment was profound, and apparently resulted in measurable, observable change in: ID knowledge development, extension and refinement of ID process skills, increased range of application, improved transfer flexibility, more adaptive use of systematic design (vs.content knowledge or intuition), and selective and strategic transfer.Changes in beliefs and identity were also profound, and mapped back to many of those same characteristics that apparently promoted skill development.
Among individual differences and preferences, those who made the choice to stretch outside of the comfortable and familiar, beyond the range mandated by the instructor, gained more than their peers who chose not to stretch.This stretching produced development across a wider range of ID component skills and application to more content areas.Those who developed more dramatically were equipped with a cluster of characteristics including: greater self-efficacy, preference for cognitive challenge, tolerance for risk and uncertainty, receptivity to experiential learning, mastery goals, and receptivity to critique and feedback.These characteristics promoted a host of related benefits that supported development, including choosing more and varied tasks, inquiry leading to understanding, less perfect work initially but more productive exploration, greater creativity and unconventional design, deeper metacognition and productive self-regulation.

Contribution to the Field of ID
This study contributes to knowledge in the field in a number of ways.It examined how individual differences and perceptions interact to drive the development of expertise differentially for learners (per Bereiter & Scardamalia, 1993;Goel & Pirolli, 1992).It included multi-source and mixed methods data for both process indicators and product outcomes, taking into account individual learner differences, such as need for cognition (Cacioppo, Petty & Kao, 1984), need for structure (Newburg & Newson, 1993) and self-efficacy (Bandura, 1997).It extended the study of development in ID competencies over a longer period than most previous studies, and included its situatedness in the design of learning environments (Anderson, Reder & Simon, 1996;Bransford, Brown & Cocking, 1999;Brown, Collins & Duguid, 1989;Chi, 2006;Dai & Sternberg, 2004).
Unlike some previous research (e.g., Tabernero & Wood, 1999), this study found task-specific self-efficacy strongly related to self-regulation (as self-evaluation and monitoring).This may be in part due to the participant group, which unlike many previous studies, were not teachers but professional design students.Building on previous research (e.g., Hardré, Ge & Thomas, 2007), it added more detailed assessments of perceptions, and revealed a cluster of perceptions that apparently position learners for a greater success in learning ID.Beyond confirming previous findings (generally from single-semester studies; e.g., Hardré, Ge & Thomas, 2007), this design extends the timeline of development to two semesters, enabling systematic assessment and analysis over a longer developmental trajectory.
Beyond the findings of other two-semester studies (e.g., Ge & Hardré, 2010), the present study introduced more robust assessments and a more coherent and diverse sample for studying the nature and role of community.Like previous studies with professional design students (e.g., Ertmer & Cennamo, 1995;Quinn, 1994), the present study notably features peer interactions, and the role of the instructor as expert model and coach.However, this study added tools drawing from multiple paradigms of ID not addressed in those earlier studies, seen through richer data sources (e.g., observations, interviews).Examples here include the nature and role of deliberate practice (Ericsson, Krampe & Tesch-Römer, 1993;Fadde, 2009), the role of problem/task complexity (Ericsson, Rikers & Paas, 2005), the role of metacognitive processing (Flavell, 1991), and the linkages between educational activities and ID practice articulated in the learning environments (Quinn, 1992;Rose, 2002).The complex data enabled us to unlock some of the "how" and "why" of design features that contributed to the development of both demonstrated and perceived ID competence.

Implications for Research and Design Education
This study addresses some characteristics of the learning environment for ID and how they align with demands of the professional field.These findings inform and invite additional research, to further investigate how these characteristics individually and interactively contribute to competence development.This study moves toward defining some characteristics of learners and learning environments that promote efficient and effective development of ID knowledge and skills.One example of such characteristics is the individual difference profile that positions novices for success in ID skill development, related to the nature of the field, its content and practice.This profile is linked to previous research findings (both in and beyond ID) and addresses links between learner and environmental characteristics (such as the cluster of individual and perceptual characteristics with features of learning environment design), and between learner background and the field of ID.
Related to the issue of autonomy in applying and integrating feedback into development of design, a question that arises is how the novice designers integrate information across phases and types of input, demonstrating iterative progress.Theoretically, each design should begin on a higher skill level, but the apparent trajectory of these novice designers was not as linear as expected.Over the study period, questions emerged on how pivotal characteristics of learners mapped onto evidence of competence across key process and product indicators.Even closer tracking of participant activity and systematic assessment of multiple versions of design products can provide clearer evidence of competence development to illuminate these issues.
The field of ID needs more systematic, context-sensitive research on competence development.This future research needs to draw on related work in educational psychology, to consider instructional components and assessment tools largely neglected to date.Such a body of progressive research will support improved design of both traditional and digital learning environments intended to develop ID skills and practice.
Beyond ID alone, these findings bear implications for other similarly complex applied design fields, such as engineering and architecture.They illuminate how learners' individual differences interact with the nature of the disciplines and tasks, as highlighted in the learning environment, to produce insights on how and why certain students develop more readily or rapidly than others.
Colleagues may wish to test these findings in related disciplines and similar fields of professional practice.
The questionnaire packets include demographics and a set of perception and response instruments.Demographics included age, gender, ethnic group, educational background, degree program and status, general work experience and background.The set of perception and descriptor scales focused on the students' individual characteristics, perception of learning and instructional design application and ability.The eight parts of the questionnaire measured goals, need for structure, need for cognition, perceived design competence, design efficacy, ID knowledge and strategies, satisfaction with ID knowledge and strategies, and reflection on the environment with attributions for development.Instrument responses (except the open-ended qualitative knowledge assessment) were on Likert-type 6-point scales (anchored from 1= Not at all to 6=Strongly Agree; or from 1=Not at all true to 6=Very true).All questionnaire instruments and rubrics were previously tested in this population (ID graduate students) and demonstrated satisfactory internal consistencies (as Cronbach's alphas of .70 or greater), as well as consistency with multiple other measures of the same outcomes (demonstrating validity)