This exploratory study, conducted at a higher education school in Mexico, investigates whether Artificial Intelligence (AI) can ensure consistency, validity, and reliability in the assessment of learning artifacts. The research emulated the human peer assessment process, traditionally conducted by evaluators and a score breaker, by applying three AI systems (two local and one open-access) to evaluate competencies in proposal design and self-reflection. The results showed that both local AI models generated similar scores aligned with the rubric criteria, while the open model showed a slight deviation. However, when compared to human assessments, the AI tended to overestimate performance, favoring textual agreement with the rubric descriptors over conceptual depth or reflective reasoning. These findings indicate that while AI can demonstrate procedural consistency, it still lacks the interpretive sensitivity necessary to assess complex cognitive and metacognitive dimensions. Methodologically, the study underscores the importance of calibrating prompts, integrating more specific institutional rubrics, and conducting iterative testing to improve evaluative validity. From an ethical perspective, it raises questions about transparency, bias, and the appropriate role of AI in educational assessment. Future research will seek to develop an experimental framework that systematically compares human and automated assessments, moving toward reliable, equitable, and pedagogically sound models of AI-assisted assessment in higher education.

Authors:
Nora Gavira-Durón, Universidad de las Americas Puebla, Mexico
Claudia María Ramirez-Culebro, Universidad de las Américas Puebla, Mexico
Angélica Alonso-Rivera, Universidad de las Américas Puebla, Mexico

About the Presenter(s)
Dr. Nora Gavira Durón, Head of Women in STEM Entrepreneurship (WISE) Latin America at UDLAP, researches AI, finance, gender equity, and education. Her current project focuses on gender gaps and digital innovation in STEM education.

Connect on Linkedin
https://www.linkedin.com/in/nora-gavira-dur%C3%B3n-31a80b110

Additional website of interest
https://orcid.org/0000-0002-7850-7966

See this presentation on the full schedule – Saturday Schedule

Bookmark