Generative AI affects the assessments in the courses we teach. The assignments we have used for a long time may no longer seem sustainable with ChatGPT. How can we ensure our students have achieved the course learning outcomes in the era of generative AI? What assessments could we use to effectively assess student learning?
In addition, students may need structure and assistance as they use generative AI tools like ChatGPT or Claude in their work. How can we best serve and support students before and after they submit assignments? This resource will explore and discuss designing and implementing assessments in the age of AI.
Designing Assessments
Generative AI calls into question and complicates the process of assessing student learning: The content a student submits may not reflect their learning but rather may reflect the output generated from ChatGPT or another tool. However, there are other ways we can assess and measure student learning, other ways we could possibly use to determine if students have met our learning outcomes or objectives. As ChatGPT and other services play an increased role in our world, we may like to either modify our assignments or use new types of assignments altogether.
Please note that the suggestions below assume a course policy that allows at least some use of generative AI (the policy of the University of Missouri System is that students cannot use generative AI unless permitted to do so by their instructor).
Updating Existing Assessments
The existence of generative AI does not mean you need to reinvent the wheel. Many of the assessments you have used in the past will still work with some tweaks.
A concern with ChatGPT and other tools is the use by students for much if not all of an assignment. However, such a concern is potentially flawed: It does not recognize that the student may have needed scaffolding and support in preparing for and doing that assignment. By approaching assignments as a process rather than as a product, we can give students the structure they may need to thrive (Murray, 1997).
Please consider infusing into your teaching various activities for all stages in the composition process: brainstorming, organizing, drafting, revising, and editing. Focusing on the process may lower the stakes for students, increase their engagement and success, demonstrate your investment in their learning, and reduce the incentive to cheat.
An instructor could go further and include activities and formative assessments that use generative AI during the writing process. Through thoughtful and well-designed instruction, students could engage with, evaluate, and build on thesis statements, outlines, or feedback from ChatGPT. Students could receive structured opportunities to use generative AI in effective, thoughtful, and meaningful ways to produce their work. For example, an instructor could ask students to use a tool like Claude to outline the evidence for dark matter. Students would then need to engage in further research to better understand and explain this evidence.
What is important is not that students are doing their work on their own, but rather that they are exercising agency and taking ownership of their learning. We argue that what matters is this: not that students do not use ChatGPT, but rather that students still critically engage with the material. By including more formative assessments and by using process-centered teaching (Li, 2009), you can build opportunities for students to critically engage with the material even if they are using generative AI.
While students may turn to generative AI, they should not relinquish control over their learning or cease engaging with the content. In addition, they should remain conscientious and deliberate in their use of generative AI tools like ChatGPT. To support this, instructors should give students the time and space to process and reflect on their use of generative AI.
Offering these avenues will invite students to identify and share strategies for effective use of ChatGPT and other tools. Through reflections, students can also explore how these tools are impacting their learning. These reflections will illuminate for the instructor the ways in which students may have used generative AI and whether they are potentially depending on it too much.
Along with these reflections, the instructor may like to obtain other documentation from the student about how they have used generative AI. This can include screenshots of prompts and outputs used for an assignment or even links to relevant chats. This documentation will further clarify for the instructor how students are using generative AI and if they need further support or guidance.
It may seem as though generative AI can discuss anything. After all, students could even upload a peer-reviewed academic journal article to Claude to obtain a summary to use in an annotated bibliography! One may naturally worry that students can offload much of the process of writing to these tools. Generative AI is becoming increasingly capable of gathering and discussing secondary research. Note, however, that generative AI cannot do its own primary research.
Therefore, you may like to consider requiring students to conduct their own primary research as part of an existing assignment. If, for example, you expect students to write a research paper, you could require them to gather their own data to corroborate or enrich the information from other sources. If your students are expected to conduct interviews or surveys, to perform an experiment, or to do an observation, they must produce content and data that generative AI cannot provide. Of course, students may fabricate their own data, but instructors can prevent this through scaffolding and well-structured activities and guidelines.
A recommendation often given is that faculty could also incorporate recent events, which allegedly are beyond the scope of the LLM (large language models). However, the technology is rapidly evolving in this regard: Already some tools like Bing and ChatGPT can incorporate and use live internet searches in some capacity.In addition, asking students to discuss very recent events may challenge generative AI, at least as the technology currently stands: The LLMs (large language models) that support generative AI tools may not have received updates yet to include the week’s news. Asking students to connect course content to these events may also challenge them to step beyond turning to generative AI. Please note that the technology is rapidly improving in this regard, and this recommendation may not prove as viable in the future.
Changing Assessment Types
We can tweak existing assignments to accommodate generative AI. In addition, we can shift our assignments altogether, using different approaches, genres, and content. Though the content may differ, students will still achieve the same learning outcomes. According to Missouri Online, using a variety of assessment types enhances learning and provides alternatives to lecturing. We can use authentic assessments (also called alternative assessments) to ensure students have still met our learning outcomes.
Using authentic assessments may encourage students to make more thoughtful or limited use of generative AI. For some students, generative AI could serve as a support system for assignments and material that do not interest them. However, authentic assessments could mitigate this problem through their increased relevance and creativity and the increased motivation they encourage. Though it is possible if not likely that students may still turn to generative AI as part of their workflow, an instructor could design meaningful learning activities to scaffold learning. Using these activities will encourage students to build upon and partner with, rather than depend on, generative AI in their work.
Implementing Assessments
When you implement your new assessments, students will need support, structure, and empathy to complete these assignments. In addition, even after submitting their work, students may need guidance and assistance in understanding how to properly use generative AI in your specific course and beyond.
- Reinforce and clarify expectations for each individual assignment. Though you may have already introduced your policies previously in a course, take a moment to engage students in a conversation about how these policies fit in the context of a specific assignment. For example, students using an image creator (DALL-E, Stable Diffusion, Midjourney, etc.) for presentations may need guidance on citing AI-generated images in MLA style.
- Give examples and discuss them. If you allow students to use generative AI to an extent on an assignment, take some time to model, review, and discuss such use. For example, if you allow students to use generative AI to create arguments and counterarguments on an issue as a starting point for a research paper, model using background knowledge, reliable online sources, and subject matter experts to expand on and enrich this discussion.
- Invite students to share their ideas and plans. You may have an assumption about how students could best use generative AI for an assignment. However, students may see other opportunities and ways they can use generative AI in their work. You may like to offer students the chance to discuss and share how they plan to use generative AI for an assignment (within the bounds of your policies) to explore new uses of the technology.
- Check in on students. Engaging in formative assessments through CATs (classroom assessment techniques)(Vanderbilt University, 2023) or one-on-one conferences can give us further insight into how students are approaching and engaging with their work on an assignment. By checking in on students, you can prevent and avoid needing to have a conversation about the use of generative AI after the assignment is completed and submitted.
After Submission by Students
When students complete and submit an assignment, your work is not over: Students may still need support and guidance with using generative AI. To help students build on this experience and apply their learning in the future, please consider integrating the following into your teaching:
- A discussion or reflection on the experience of using generative AI: Invite students to participate in a conversation with classmates or reflect privately in a journal. Regardless of the format, ask students to describe their experience of using generative AI. How did they use it on the assignment? How did generative AI help them? What challenges did it present? What strategies for using generative AI have they learned that they may like to use in the future?
- An opportunity for feedback from students: While you may give students feedback on their work, it is important for them to give your feedback as well. After all, they may have ideas and suggestions that could only improve the learning experience in the future. Ask students, either in discussions or through surveys, what was effective or ineffective with the assignment and how they could use generative AI with it. What other ways could students have done the assessment? What would offer a more effective, efficient, or engaging experience for future students?
By taking these steps before and after assignments, you can ensure students thrive in using generative AI as part of their learning experience.
If you have any other questions or concerns about how to use generative AI in your teaching or to discuss ways you could use it, please reach out to Missouri Online at teachingonline@umsystem.edu.
What if you suspect a student has used generative AI in their work on an assignment too? Please consult our article, Detecting Artificial Intelligence (AI) Plagiarism.
Related sprints
If you would like to learn more about creating effective assessments, please consider enrolling in one of our Sprints!
Sprint 2.A Learning outcomes: Edujargon or educessity?
Learning outcomes serve as the road map for a course. In this sprint, you will use Bloom’s Taxonomy and the SMART technique to create or revise course- and module-level learning outcomes.
Sprint 2.B Active learning
Active learning is an instructional method that engages students in the learning process beyond listening and passive note taking.
Sprint 2.C Authentic pedagogy in the age of AI
Learn to craft meaningful assignments that resonate with real-world scenarios in the brave new world of authentic intelligence (AI).
Sprint 2.D Academic integrity
This sprint discusses how to create a learning environment that supports academic integrity in any modality.
References
Li, B. (2009). Process-centered teaching and its implications in English teaching in China. English Language Teaching, 2(1), 24-30.
Murray, D. (1997). Teaching writing as a process not product. In V. Villanueva, Jr. (Ed.), Cross-talk in comp theory: A reader (pp. 3-6). National Council of Teachers of English.
Northern Illinois University Center for Innovative Teaching and Learning. (2012). Instructional scaffolding to improve learning. https://www.niu.edu/citl/resources/guides/instructional-guide/instructional-scaffolding-to-improve-learning.shtml
Created on: November 30, 2023