Risk and Reward: Integrating GenAI into Educational Assessment

As GenAI improves, ideas of authorised or unauthorised use in assessment become harder to discern. Sarah Grundy, Peter Neal, and Sasha Nikolic suggest the controls the community need to use to ensure educational assessment remains secure

CHEMICAL engineering educators must respond positively to the hazard that GenAI poses for assurance of student learning. But by using a qualitative risk analysis approach to evaluate GenAI in assessments, we can mitigate it using the same hierarchy of controls we use to mitigate process risks.

To evaluate the risk, we first defined a likelihood scale ranging from almost certain that GenAI will be misused every time that assessment type is used through to rare occurrences where GenAI misuse occurs perhaps once per degree or is only theoretically possible.

Then we set levels of consequence to measure the impact on assuring student learning of GenAI misuse. Incidences of insignificant or minor consequence have no impact on assurance of learning in themselves or because the learning is assured elsewhere. Whereas major or severe levels of consequence would raise significant doubt or negate assurance of student learning.

Using a 5 x 5 risk matrix (shown above), we were then able to estimate the risk of various assessment types. In our paper we present a matrix of assessment security and integration opportunity options that both provides our risk ratings for each assessment type but also explains short and long-term security options, as well as opportunities for incorporating GenAI into the task. The full risk assessment process is explained here.

The following table illustrates how different controls can be used.

LABEL	DESCRIPTOR
Eliminate (Remove)	Remove the need for this assessment (type). For example, given the high risks associated with take-home exams, an educator may redesign the assessment suite in a unit to eliminate the need for a take-home exam
Substitute (Replace)	Replace this assessment with a lower risk task/type. For example, replacing a report or essay task with an interactive oral assessment reduces the likelihood of misuse of GenAI as well as the consequence since the new assessment provides much greater assurance of learning
Engineering (Redesign)	Change the design or weighting of the assessment to control or incorporate the use of GenAI. For cognitive competencies, shift towards more authentic assessment types that require strong evaluative judgment. This includes creative assessments that, even with the aid of technology, passing is no easy task. Take advantage of experimentation and reevaluate the importance of psychomotor and affective skills which fall out of GenAI reach. Alternatively, embrace the potential of GenAI as a teaching aid rather than viewing it solely as a threat. Use GenAI tools to create innovative learning materials, provide personalised feedback, and support students in their learning journey
Isolate (Relocate)	Remove or practically limit the ability of students to use GenAI with the assessment task in unauthorised ways. This approach may include using in-person exams, conducting assessments in class, using GenAI under invigilated conditions, or invigilated quizzes with Safe-Exam Browser
Administration (Remind)	Explain whether/how students may use GenAI with the assessment task. This may include institutional policies and guidance (eg University of Sydney two lanes, UNSW permitted use categories, AI Assessment Scale), as well as educating students on appropriate use of GenAI. Assessment integrity goes well beyond the subject coordinator and requires an institution-wide approach to be effective and to enable ethical learning. Administrative controls are crucial for establishing this ethical culture. However, simply stating or asking students not to use GenAI without effective and accurate detection and enforcement mechanisms are counterproductive
Personal (Restrict)	Measures to control or detect the use of GenAI student by student. These approaches are the least effective and potentially the most time consuming. Some examples of personal controls including reviewing document logs of submitted files, and AI detection tools

Recommendations for students

Students also have a role to play in maintaining academic integrity while leveraging GenAI for their benefit:

1. Use GenAI as a Learning Tool: Approach GenAI as a resource to enhance understanding and aid in studying. Use it to clarify concepts, explore different perspectives, and practice problem-solving rather than as a means to complete assignments dishonestly. For example, maths tutoring has been difficult and expensive to obtain, and if the correct tool is used, it is getting rather good. Remember, by not cheating, you are providing evidence of the competency of ethical conduct
2. Develop Critical Skills: Focus on building strong foundational knowledge and critical thinking skills. Engage deeply with the course material and participate actively in discussions and practical sessions. To use GenAI to its maximum potential, one needs to have a strong evaluative judgment (a strong understanding of the content). Without it, GenAI use can be dangerous and anti-productive
3. Seek Support When Needed: If struggling with coursework, seek help from instructors, peers, or academic support services rather than resorting to GenAI for quick fixes. Utilise GenAI to supplement learning under the guidance of educators

Adapting to the future

The rise of GenAI presents both challenges and opportunities for chemical engineering education. By understanding the risks and leveraging the benefits, educators can adapt their approaches to enhance learning while safeguarding academic integrity. Departments must act swiftly to audit and strengthen their anti-cheating measures, ensuring that the rapid advancements in GenAI do not undermine the quality and credibility of engineering education.

In this dynamic landscape, the key lies in balancing the integration of GenAI as a powerful educational tool with the maintenance of rigorous academic standards. Through innovative assessment design, ethical GenAI use, and a strong emphasis on integrity, the chemical engineering community can navigate these changes effectively, preparing students for a future where technology and engineering excellence go hand in hand. A student who leaves university with strong content knowledge, well-formed evaluative judgement, rounded psychomotor and affective skills, and capable of integrating GenAI into their work to enhance efficiency will be a student ahead of the pack.