We will now turn to designing an assessment that is authentic, accessible, learner-centred and realistic for you to implement in the context of your course.
There are at least two ways to engage with the module:
- as a set of considerations to reflect upon, or
- as a practical workbook to take you through the design of an assessment (please capture your thoughts when prompted)
If you choose the route of recording your responses, we have created an exportable workbook to guide your design of an authentic assessment, including a series of questions to guide your assessment design. The written content from the workbook is repeated below if you would prefer to simply read and reflect without answering the questions.
Some questions may be more useful at the beginning of your assessment design process, while others may be easier to answer with at least a preliminary concept in mind. If you would like to sample some non-traditional assessment ideas before you get started, you may want to jump ahead to the Ideating section of the module first and return to the workbook once you have an assessment idea to work with.
Jump to assessment design workbook [link to anchor for H5P interaction in Appendix(?)]
The content in this “Designing Authentic Assessments” tab is based on an open educational resource (OER) developed at the MacPherson Institute, Generative Artificial Intelligence in Teaching and Learning at McMaster. The section about designing assessments in the age of generative AI was co-authored by Joanne Kehoe and Devon Mordell.
Taking Stock of Assessment Constraints
Before exploring some of the principles of assessment design, you may wish to begin by documenting a few details about the overall context of your course assessments that might impact their design.
Learning outcomes state what learners are expected to be able to know, do, or value at the end of a class, unit, course or program. For example, “by the end of the workbook, learners will be able to evaluate the (constructive) alignment of their assessments with their course learning outcomes.”
You have likely been asked to write course-level learning outcomes when preparing a syllabus. You may also be aware of the program-level learning outcomes in your department… perhaps you even contributed to their design!
The central feature of a learning outcome is a specific and measurable action verb that implies how to assess the learning outcome, such as “solve,” “compose” and “justify”; verbs like “understand,” “learn” and “know” are typically avoided in learning outcomes as they are imprecise and difficult to observe. In the earlier example of an outcome about learners being able to evaluate the alignment between assessments and outcomes, we might present the learner with a small set of learning outcome / assessment pairs and ask them to discuss whether they are aligned.
Because of the connection between learning outcomes and assessments, some verbs are more prevalent in certain disciplines compared to others – like “calculate” or “critique.” Which action verbs describe competencies within your own discipline?
Consider:
Which course learning outcome(s) could or should the assessment measure? Put another way, if a student successfully completes the assessment, what learning outcomes will they have demonstrated? Will students understand why they are doing what they are doing?
Are there any program-level learning outcomes that students can show they have met in completing the assessment?
Certain factors or teaching conditions may limit the possibilities for assessment design, so it is helpful to identify them early on.
Consider:
What design constraints must be accounted for in your assessment?
- Class size?
- Available TA support for grading?
- Other assessments in the course?
- Student access to technology / internet?
- Other?
Some disciplines may be required to align assessments with accreditation guidelines; move on to the next question if this is not true of your discipline.
Consider:
Does the assessment need to conform to any accreditation requirements? If so, what are they?
You may already have an assessment that you have found effective, but you need to adjust it due to a change in assessment circumstances – like the availability of generative AI, or a larger class size.
If not, read on: “Defining Authentic Assessment.”
Redesigning an Existing Assessment: Strengths
Consider:
What are the strengths of the existing assessment? What positive feedback about it have you received from students?
What course or unit learning outcome does it demonstrate that students have achieved? What knowledge, skills or attitudes does it develop in learners?
Redesigning an Existing Assessment: Reasons for Redesign
Consider:
What are your reasons for wanting to redesign the assessment? Have students made specific comments about it in your course evaluations?
How might the assessment be vulnerable to academic integrity issues, whether related to developments such as generative artificial intelligence or not? Are there a significant number of accommodation requests that suggest a redesign might benefit you and your students alike?
Defining Authentic Assessment
This “Rethinking Assessment” module centres on the idea of authentic assessments, which are meant to “foster disciplinary behaviours and ways of thinking and problem solving used by professionals in the field” (Queen’s University Centre for Teaching & Learning, n.d., para 3). The definition of authentic assessment is not widely agreed upon, however, leading it to become a catch-all term for good assessment practices in general. Because of its openness to interpretation, we will state our understanding of the term – based on the work of Grant Wiggins (1998, via Indiana University Bloomington) – below.
“An authentic assessment:
- is realistic.
- requires judgment and innovation.
- asks the student to “do” the subject.
- replicates or simulates the contexts in which adults are “tested” in the workplace or in civic or personal life.
- assesses the student’s ability to efficiently and effectively use a repertoire of knowledge and skills to negotiate a complex task.
- allows appropriate opportunities to rehearse, practice, consult resources, and get feedback on and refine performances and products. “
(Indiana University Bloomington, n.d., para. 2)
Authentic assessments typically draw on complex real-world situations and constraints in their design.
Consider:
Who are the practitioners in your field? What do they know and do as part of their work or research?
What might the future of your discipline look like, in response to developments such as generative AI or other challenges specific to your area of study?
In what ways does or could the assessment offer opportunities to “do” their field of study?
Consider:
What tasks are customary in your disciplinary area?
Are there ways you can simulate the context of a workplace, civic or personal life in certain respects? What real-world constraints could an assessment involve?
Authentic assessments are multifaceted and complex. They demand that students use a range of skills and knowledge in coordination, and there is usually more than one right answer.
Consider:
What skills do students develop in the course? How might the assessment require students to apply those skills in ways that relate to other skills and knowledge they gain through your course?
Will assessment products or performances allow for a variety of responses or approaches that are equally valid, if appropriate?
Authentic assessments require more than just correct answers: they require judgment, innovation and justification on the students’ part.
Consider:
How might your assessment ask students to exercise their judgment – evaluate, critique or otherwise apply what they know in novel scenarios? Will the assessment allow for creativity on the student’s part?
If so, how? If not, what kinds of authentic activities or tasks could be incorporated into the assessment?
Rather than being a “one-shot” task, authentic assessments involve students getting feedback they can use to refine assessment products and performances, and apply particular knowledge or skills in different ways or contexts.
Consider:
Can formative feedback be built-in to the assessment? Will students have the ability to revise and refine their submissions?
Will students know or be able to predict assessment tasks and standards for judgment in advance? Can assessment tasks be iterative, giving students the chance to improve and grow their understanding of the knowledge and skills they are developing?
Students from outside your discipline may also be enrolled in your course – if it is helpful to your assessment design process, consider what interested or motivated them to take the course.
Consider:
For what reasons might non-program students be taking your course as an elective? What course competencies can they use or translate to their everyday lives?
Information Box Group
We appreciate reservations about framing university education as workplace training while also recognizing students’ expectations about employability following their degree. Wiggins’ definition finds common ground between the two concerns: Wiggins sees the “‘ill-structured’ challenges and roles” of authentic tasks as a rehearsal for the complexities of “adult and professional life” (1).
Other Assessment Design Principles
While authentic assessment is a helpful concept to frame our thinking, it does not fully capture the needs our assessment design process. For example, it is also important that we design our assessments with accessibility in mind and ensure that they are aligned with what we want our students to learn. Some other considerations are listed below, and you may have additional ones that are tailored to your teaching context.
From Biggs and Tang, the principle of constructive alignment combines the idea that “learners use their own activity to construct their knowledge as interpreted through their own existing schemata” and “assessment tasks should be aligned to what it is intended to be learned” (2011, p. 97).
The principle of constructive alignment is operationalized through the practice of backwards course design, where you start with the end in mind (i.e. the learning outcome, what you want students to be able to do by the end of the course) and work backwards to determine how to support students in reaching that end through the design of assessments, activities and course content.
Consider:
Referring back to the learning outcomes you identified in the “Taking Stock of Assessment Constraints” section of the module, what kinds of activities does the verb in your learning outcome(s) suggest? For example, what kinds of assessments would allow you to measure whether students are able to hypothesize, analyze or critique?
When you have determined that your assessment demonstrates students have met course or program-level learning outcomes, ask yourself: what course activities and content will help students to complete the assessment?
Ensuring that your assessment is accessible to the greatest extent possible for all learners is an important consideration from the outset of assessment design. The universal design for learning (UDL) guidelines offer one approach to improving the accessibility of your assessments for all learners, with recommendations such as building in flexibility, sharing rubrics to clarify expectations and providing mastery-oriented feedback.
Consider:
Are there any potential barriers for students within the assessment? For example, are there graphs that would need to be described for students who are blind or who have low vision? Are there aspects of the assessment that may disadvantage students who are English-language learners, such as lengthy and complex instructions?
Is it possible for students to be provided with multiple options to demonstrate their abilities and learning? That is, can the assessment exemplify the UDL principle of multiple means of action and expression?
How will you communicate your expectations around the successful completion of the assessment? Will there be opportunities for students to receive feedback on their work before submitting a high-stakes assessment?
Learner-centred instruction is grounded in a constructivist theory of learning, where students are recognized as having an active role in the construction of their own knowledge rather than being passive recipients of content.
In its fullest expression, “[s]tudent-centered learning demands that students set their own objectives for learning and determine the resources and activities that will help them meet those objectives” (Jonassen, paraphrased in Duncan & Buskirk-Cohen, 2011, p. 247). While it may not be possible in most teaching contexts for students to design their own assessments, consider incorporating elements of choice and scaffolding (e.g. breaking a large project into stages or sharing approaches to studying) to build students’ confidence in their ability to learn.
Through an equity lens, learner-centredness also recognizes the diversity and individuality of learners; that is, students are able to see themselves reflected in the assessment.
Consider:
How might the assessment acknowledge the lived experience of the learner? I.e. Will the assessment allow students to draw on their own background, circumstances, strengths, and interests?
In what ways will the assessment offer tailored opportunities for learners to show what they know and what they can do? Can the assessment support the development of autonomous, self-directed learners?
When appropriate, assessment design can offer an opportunity to develop students’ digital and AI literacy. It is important, however, to consider whether incorporating AI might distract from the learning outcomes that the assessment intends to assess. It should be students who execute the key actions at the heart of the intended learning outcomes, thus demonstrating their competency in those areas (with the technology merely playing a support role).
Definitions of what constitutes AI literacy are in flux, but Long and Magerko (2020) broadly capture the idea as “a set of competencies that enables individuals to critically evaluate AI technologies; communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the workplace” (p. 2). Commonly cited competencies include writing effective prompts, assessing the correctness of facts referenced, having a general understanding of how AI works and evaluating for bias.
Consider:
In what ways might the assessment prepare students for an AI-integrated future? If generative AI is integrated into the assessment, does it encourage students to reflect and be critical about their use?
Could any generative AI components overshadow or take away from the original goals of the assessment?
References
Biggs, J. B., Tang, C. S. (Eds.) (2011). Teaching for quality learning at university: What the student does (4th ed.). McGraw-Hill, Society for Research into Higher Education & Open University Press.
Duncan, T., & Buskirk-Cohen, A. A. (2011). Exploring Learner-Centered Assessment: A Cross-Disciplinary Approach. International Journal of Teaching and Learning in Higher Education, 23(2), 246–259.
Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 598–598. https://doi.org/10.1145/3313831.3376727
Mordell, D., & Kehoe, J. (2023). Part 2: Redesigning Your Assessment. Paul R. MacPherson Institute for Leadership, Innovation and Excellence in Teaching. https://ecampusontario.pressbooks.pub/mcmasterteachgenerativeai/chapter/part-2-redesigning-your-assessment/.
Wiggins, G., (1990) “The Case for Authentic Assessment,” Practical Assessment, Research, & Evaluation, 2(1): 2. https://doi.org/10.7275/ffb1-mm19.