Comment ChatGPT note-t-il le travail ? Voyons ça avec Daisy Christodoulou.

Table of Contents

1. Introduction
2. What is No More Marking?
3. Comparative Judgment: A Holistic Approach to Assessing Writing
4. The Challenges of Traditional Writing Assessment
5. The Role of AI in Writing Assessment
6. Exploring Chat GPT and AI Marking Systems
7. The Inconsistencies and Limitations of AI Marking
8. The Future of AI in Writing Assessment
9. The Impact on Teacher Roles and Workloads
10. Ethical and Fair Use of AI in Writing Assessment
11. Conclusion

Introduction

In this article, we will Delve into the world of writing assessment and explore the role of AI in marking student essays. We will specifically focus on No More Marking, a provider of online comparative judgment assessments, and discuss the innovative technique of comparative judgment that offers a more holistic approach to assessing writing. We will also examine the challenges of traditional writing assessment methods and the potential benefits and limitations of AI in this field. Additionally, we will address the impact of AI on teacher roles and workloads, as well as the ethical considerations surrounding the use of AI in writing assessment.

What is No More Marking?

No More Marking is a company that specializes in online comparative judgment assessments. Comparative judgment is a unique assessment technique that allows for a more holistic and innovative approach to assessing writing. Unlike traditional methods where writing is assessed against a rubric, comparative judgment involves scanning students' writing into the system and presenting pairs of pieces of writing on-screen. The evaluator then determines which is the better piece of writing, making a series of decisions like this. The collective decisions of evaluators are used to establish a measurement Scale for every piece of writing. No More Marking works with thousands of schools globally, running assessments with tens of thousands of students.

Comparative Judgment: A Holistic Approach to Assessing Writing

Traditional methods of writing assessment often lack a comprehensive and nuanced evaluation of students' writing abilities. Comparative judgment addresses this issue by providing a holistic view of writing assessment. By comparing pairs of writing samples and making decisions Based on their quality, evaluators can arrive at a more comprehensive measurement scale for each piece of writing. This approach allows for a more accurate and reliable assessment that takes into account various aspects of writing, such as structure, coherence, and creativity. Comparative judgment not only eliminates the limitations of traditional rubric-based marking but also frees evaluators from the burden of subjectivity and inconsistency.

The Challenges of Traditional Writing Assessment

Traditional writing assessment methods present several challenges that hinder accurate and reliable marking. One significant challenge is the lack of consistency among evaluators. Even when using the same rubric, different evaluators may arrive at different marks for the same piece of writing. Moreover, individual evaluators can even disagree with their own decisions when asked at different points in time. This inconsistency undermines the reliability of traditional writing assessment and makes it difficult to establish an accurate measurement scale. Another challenge is the built-in margin of error, which leads to grading discrepancies and can negatively impact students' educational journeys.

The Role of AI in Writing Assessment

AI has been touted as a potential solution to the challenges of traditional writing assessment. By utilizing AI, it is believed that writing assessment can become more consistent and reliable. However, the effectiveness of AI in the marking process is still a topic of debate and further exploration. The integration of AI into writing assessment systems offers the potential to enhance efficiency, speed, and scalability. It could potentially reduce teacher workloads, foster standardization, and provide valuable insights into student performance. However, the reliability and accuracy of AI marking systems need to be thoroughly examined before widespread implementation.

Exploring Chat GPT and AI Marking Systems

Chat GPT is an AI language model that has garnered significant Attention in the field of writing assessment. No More Marking conducted trials using Chat GPT to explore its potential as an AI marking system. However, the results were not as promising as initially expected. The reliability of Chat GPT was found to be lacking, with inconsistent marks and questionable alignment with human evaluators. Despite efforts to aggregate multiple marks and integrate predefined mark schemes, Chat GPT struggled to produce accurate and reliable results. Furthermore, there was a lack of transparency in understanding how Chat GPT arrived at its marks, further complicating the assessment process.

The Inconsistencies and Limitations of AI Marking

While AI holds the promise of consistency, it is crucial to acknowledge the limitations and inconsistencies that arise in AI marking systems. The traditional strength of AI in providing consistency is not as evident in Chat GPT. Unlike other AI models, Chat GPT does not consistently produce the same marks when given the same essay at different times. This lack of consistency raises concerns about the reliability and validity of AI marking. Additionally, Chat GPT's difficulty in adhering to predefined mark schemes and its tendency to cluster marks in the middle further highlight the challenges of using AI in writing assessment. These limitations necessitate careful consideration when implementing AI marking systems.

The Future of AI in Writing Assessment

The future of AI in writing assessment remains uncertain. While AI has the potential to revolutionize the way we assess writing, it is crucial to address the limitations and challenges associated with its implementation. The Current crop of AI models, including Chat GPT, does not provide a reliable solution for writing assessment. However, the development of future models and advancements in AI technology may offer more promising outcomes. The integration of AI into writing assessment systems should be approached with caution and thorough evaluation to ensure both reliability and fairness in the assessment process.

The Impact on Teacher Roles and Workloads

The introduction of AI in writing assessment has implications for teachers' roles and workloads. Comparative judgment, paired with AI technology, offers a quicker and more reliable method of assessing writing than traditional marking approaches. By reducing the time spent on marking, teachers can allocate more time to providing personalized feedback and guidance to students. However, there are concerns about the ethical implications and potential devaluation of teachers' expertise due to the reliance on AI systems. It is essential to strike a balance between the benefits of AI in reducing teacher workloads and the preservation of teacher-student interaction and personalized assessment.

Ethical and Fair Use of AI in Writing Assessment

The use of AI in writing assessment raises ethical considerations that must be carefully addressed. AI systems must be trained on unbiased and diverse datasets to avoid perpetuating biases in marking. Transparency in the development and functioning of AI models is crucial to ensure accountability and to avoid undue reliance on opaque algorithms. Furthermore, the role of human judgment and expertise cannot be replaced entirely by AI systems. It is essential to establish clear guidelines and standards for using AI in writing assessment to ensure fairness, accuracy, and the preservation of educational values.

Conclusion

In conclusion, AI has the potential to revolutionize the field of writing assessment. However, the reliability, consistency, and ethical implications of AI marking systems must be thoroughly examined. Comparative judgment offers a holistic and innovative approach to assessing writing, providing a more comprehensive measurement scale. While AI models like Chat GPT have been explored, their limitations and inconsistencies highlight the challenges of using AI in writing assessment. The future development of AI models and the integration of AI into writing assessment systems require careful evaluation to ensure their efficacy, reliability, and ethical use. By striking a balance between AI technology and human expertise, we can leverage the benefits of AI while preserving the integrity of writing assessment.