Principled development of Workplace English Communication part 2: Expanded Evidence-Centered Design and Theory of Action frameworks

dc.contributor.authorOliveri, Maria E.
dc.contributor.authorSlomp, David H.
dc.contributor.authorRupp, André A.
dc.contributor.authorMislevy, Robert J.
dc.descriptionOpen access article. Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International license (CC BY NC-ND 4.0) appliesen_US
dc.description.abstractBackground: In today’s rapidly evolving world, technological pressures coupled with changes in the nature of work increasingly require individuals to use advanced technologies to communicate and collaborate in geographically distributed multidisciplinary teams. These shifts present the need to teach and assess an expanded set of knowledge, skills, and attitudes, including how to communicate at work in collaborative environments using diverse forms of technology. They also present the opportunity to create novel forms of instructional materials and forms of assessment that extend the more traditionally used summative assessments to assessments used for learning and instruction. This design process can be facilitated through the use of conceptual frameworks employed to guide assessment design and development. Their use is important to support more expansive and complex design goals emerging in the design of assessments of 21st century skills such as Workplace English Communication (WEC). In this article, we reflect on an evolving WEC construct needed for today’s economy and discuss implications for expanding how we teach and assess it using formative assessments for learning. We then discuss the features of the expanded Evidence-Centered Design for learning and assessment systems (e-ECD) and Theory of Action (ToA) frameworks and illustrate their integrative application to inform the design and development of WEC training modules (or resources). We conclude with suggestions for next steps in this line of research. Questions Addressed: In reference to the e-ECD and ToA frameworks, our article addresses questions in two areas. We illustrate the benefits of using the ToA to explicitly identify the components of an assessment, its action mechanisms, stakeholders’ needs, score-based decisions and their impact, and the services designed for test takers and users. We illustrate the benefits of using the e-ECD framework to guide design efforts in principled ways to enable consideration of both key elements that relate evidentiary elements relevant to the construct, aspects of learning and assessment, and measurement models. Consideration of these frameworks is important to design assessments and make sense of the evidence for meaningful interpretation of students’ results. Conclusions: This article illustrates the application of conceptual (e.g., the e-ECD and ToA) frameworks that can be used to inform the design and development of similar modules for complex tasks of 21st century skills. This article contributes to the literature on WEC and complex assessments of hard-to-assess constructs more generally by offering a way of thinking about designing, assessing, and then evaluating the design and assessment of interactive educational modules for teaching complex communication knowledge and abilities while remaining attentive to (negative) consequences associated with the stakeholders designing, developing, and using the assessments.en_US
dc.identifier.citationOliveri, M. E., Slomp, D., Rupp, A. A., & Mislevy, R. J. (2021). Principled development of Workplace Enlish Communication part 2: Expanded Evidence-Centered Design and Theory of Action frameworks. Journal of Writing Analytics, 5, 71-108.
dc.publisherThe WAC Clearinghouseen_US
dc.publisher.institutionUniversity of Nebraska-Lincolnen_US
dc.publisher.institutionUniversity of Lethbridgeen_US
dc.publisher.institutionMindful Measurementen_US
dc.publisher.institutionSuccess Academy Charter Schoolsen_US
dc.publisher.institutionEducational Testing Serviceen_US
dc.subjectAnticipatory design frameworksen_US
dc.subjectBlended learningen_US
dc.subjectDigital modulesen_US
dc.subjectEvidence-Centered Design (ECD)en_US
dc.subjectExpanded Evidence-Centered Design (e-ECD)en_US
dc.subjectTheory of Action (ToA)en_US
dc.subjectWorkplace English Communication (WEC)en_US
dc.subjectWriting analyticsen_US
dc.titlePrincipled development of Workplace English Communication part 2: Expanded Evidence-Centered Design and Theory of Action frameworksen_US
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
1.45 MB
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
1.75 KB
Item-specific license agreed upon to submission