Principled development of Workplace English Communication part 2: Expanded Evidence-Centered Design and Theory of Action frameworks
Loading...
Date
2021
Authors
Oliveri, Maria E.
Slomp, David H.
Rupp, André A.
Mislevy, Robert J.
Journal Title
Journal ISSN
Volume Title
Publisher
The WAC Clearinghouse
Abstract
Background: In today’s rapidly evolving world, technological pressures coupled with changes in the nature of work increasingly require individuals to use advanced technologies to communicate and collaborate in geographically distributed multidisciplinary teams. These shifts present the need to teach and assess an expanded set of knowledge, skills, and attitudes, including how to communicate at work in collaborative environments using diverse forms of technology. They also present the opportunity to create novel forms of instructional materials and forms of assessment that extend the more traditionally used summative assessments to assessments used for learning and instruction. This design process can be facilitated through the use of conceptual frameworks employed to guide assessment design and development. Their use is important to support more expansive and complex design goals emerging in the design of assessments of 21st century skills such as Workplace English Communication (WEC). In this article, we reflect on an evolving WEC construct needed for today’s economy and discuss implications for expanding how we teach and assess it using formative assessments for learning. We then discuss the features of the expanded Evidence-Centered Design for learning and assessment systems (e-ECD) and Theory of Action (ToA) frameworks and illustrate their integrative application to inform the design and development of WEC training modules (or resources). We conclude with suggestions for next steps in this line of research.
Questions Addressed: In reference to the e-ECD and ToA frameworks, our article addresses questions in two areas. We illustrate the benefits of using the ToA to explicitly identify the components of an assessment, its action mechanisms, stakeholders’ needs, score-based decisions and their impact, and the services designed for test takers and users. We illustrate the benefits of using the e-ECD framework to guide design efforts in principled ways to enable consideration of both key elements that relate evidentiary elements relevant to the construct, aspects of learning and assessment, and measurement models. Consideration of these frameworks is important to design assessments and make sense of the evidence for meaningful interpretation of students’ results.
Conclusions: This article illustrates the application of conceptual (e.g., the e-ECD and ToA) frameworks that can be used to inform the design and development of similar modules for complex tasks of 21st century skills. This article contributes to the literature on WEC and complex assessments of hard-to-assess constructs more generally by offering a way of thinking about designing, assessing, and then evaluating the design and assessment of interactive educational modules for teaching complex communication knowledge and abilities while remaining attentive to (negative) consequences associated with the stakeholders designing, developing, and using the assessments.
Description
Open access article. Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International license (CC BY NC-ND 4.0) applies
Keywords
Anticipatory design frameworks , Blended learning , Digital modules , Evidence-Centered Design (ECD) , Expanded Evidence-Centered Design (e-ECD) , Theory of Action (ToA) , Workplace English Communication (WEC) , Writing analytics
Citation
Oliveri, M. E., Slomp, D., Rupp, A. A., & Mislevy, R. J. (2021). Principled development of Workplace Enlish Communication part 2: Expanded Evidence-Centered Design and Theory of Action frameworks. Journal of Writing Analytics, 5, 71-108. https://doi.org/10.37514/JWA-J.2021.5.1.03